[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:08:13 dist-test-slave-1vmg krb5kdc[30762](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:08:13 dist-test-slave-1vmg krb5kdc[30762](info): set up 2 sockets
Aug 11 15:08:13 dist-test-slave-1vmg krb5kdc[30762](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 15:08:14.876060 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.002s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:08:14 dist-test-slave-1vmg krb5kdc[30762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:08:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:08:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:08:15.023492 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:38377
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:39917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:38377
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:15.144696 30788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:15.145031 30788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:15.145093 30788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:15.149307 30788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:08:15.149403 30788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:15.149432 30788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:15.149453 30788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:08:15.149474 30788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:08:15.155340 30788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:39917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:38377
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:38377
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30788
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:15.156647 30788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:15.157774 30788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:15.167311 30795 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:15.167343 30794 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:15.167332 30797 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:15.167397 30788 server_base.cc:1034] running on GCE node
I20260811 15:08:15.168349 30788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:15.169637 30788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:15.170797 30788 hybrid_clock.cc:648] HybridClock initialized: now 1786460895170751 us; error 68 us; skew 500 ppm
Aug 11 15:08:15 dist-test-slave-1vmg krb5kdc[30762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460895, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:15.173950 30788 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:08:15.175132 30788 webserver.cc:460] Webserver started at http://127.30.8.62:44181/ using document root <none> and password file <none>
I20260811 15:08:15.175766 30788 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:15.175838 30788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:15.176057 30788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:15.178453 30788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "922d570cf0f94fd58fc91fa911dc9e08"
format_stamp: "Formatted at 2026-08-11 15:08:15 on dist-test-slave-1vmg"
server_key: "427f6ecca7bab54d1f99ed815226e9be"
server_key_iv: "07c9bbd8d14d3e1f3b04cc4a6ce78a18"
server_key_version: "encryptionkey@0"
I20260811 15:08:15.179121 30788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "922d570cf0f94fd58fc91fa911dc9e08"
format_stamp: "Formatted at 2026-08-11 15:08:15 on dist-test-slave-1vmg"
server_key: "427f6ecca7bab54d1f99ed815226e9be"
server_key_iv: "07c9bbd8d14d3e1f3b04cc4a6ce78a18"
server_key_version: "encryptionkey@0"
I20260811 15:08:15.183348 30788 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 15:08:15.186028 30808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:15.187423 30788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:08:15.187564 30788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "922d570cf0f94fd58fc91fa911dc9e08"
format_stamp: "Formatted at 2026-08-11 15:08:15 on dist-test-slave-1vmg"
server_key: "427f6ecca7bab54d1f99ed815226e9be"
server_key_iv: "07c9bbd8d14d3e1f3b04cc4a6ce78a18"
server_key_version: "encryptionkey@0"
I20260811 15:08:15.187687 30788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:15.199311 30788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:15.203107 30788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:15.203329 30788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:15.211108 30788 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:38377
I20260811 15:08:15.211122 30890 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:38377 every 8 connection(s)
I20260811 15:08:15.212211 30788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:08:15.215394 30891 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:15.221750 30891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08: Bootstrap starting.
I20260811 15:08:15.222903 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 30788
I20260811 15:08:15.223155 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:08:15.224932 30752 external_mini_cluster.cc:1428] Setting key 685544e68d909f6735b3c7ab780cc394
I20260811 15:08:15.225323 30891 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:15.226329 30891 log.cc:824] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:15.228852 30891 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08: No bootstrap required, opened a new log
I20260811 15:08:15.234325 30891 raft_consensus.cc:348] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922d570cf0f94fd58fc91fa911dc9e08" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 38377 } }
I20260811 15:08:15.234679 30891 raft_consensus.cc:374] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:15.234735 30891 raft_consensus.cc:729] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 922d570cf0f94fd58fc91fa911dc9e08, State: Initialized, Role: FOLLOWER
I20260811 15:08:15.235287 30891 consensus_queue.cc:260] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922d570cf0f94fd58fc91fa911dc9e08" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 38377 } }
I20260811 15:08:15.235476 30891 raft_consensus.cc:388] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:08:15.235603 30891 raft_consensus.cc:482] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:08:15.235721 30891 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:15.237097 30891 raft_consensus.cc:504] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922d570cf0f94fd58fc91fa911dc9e08" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 38377 } }
I20260811 15:08:15.237526 30891 leader_election.cc:302] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 922d570cf0f94fd58fc91fa911dc9e08; no voters:
I20260811 15:08:15.237991 30891 leader_election.cc:288] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:08:15.238098 30898 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:15.238350 30898 raft_consensus.cc:686] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08 [term 1 LEADER]: Becoming Leader. State: Replica: 922d570cf0f94fd58fc91fa911dc9e08, State: Running, Role: LEADER
I20260811 15:08:15.238772 30898 consensus_queue.cc:237] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922d570cf0f94fd58fc91fa911dc9e08" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 38377 } }
I20260811 15:08:15.239394 30891 sys_catalog.cc:564] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:08:15.240634 30900 sys_catalog.cc:455] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "922d570cf0f94fd58fc91fa911dc9e08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922d570cf0f94fd58fc91fa911dc9e08" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 38377 } } }
I20260811 15:08:15.240772 30900 sys_catalog.cc:458] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:15.241235 30901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 922d570cf0f94fd58fc91fa911dc9e08. Latest consensus state: current_term: 1 leader_uuid: "922d570cf0f94fd58fc91fa911dc9e08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "922d570cf0f94fd58fc91fa911dc9e08" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 38377 } } }
I20260811 15:08:15.241397 30901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:15.242386 30916 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:08:15.246205 30916 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 11 15:08:15 dist-test-slave-1vmg krb5kdc[30762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460894, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:15.253878 30916 catalog_manager.cc:1347] Generated new cluster ID: 6d11a162fef74ef39e9c2330cec65572
I20260811 15:08:15.254046 30916 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:08:15.254737 30896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:15.228931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54952 (local address 127.30.8.62:38377)
0811 15:08:15.229455 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:15.229476 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:15.230818 (+ 1342us) server_negotiation.cc:407] Connection header received
0811 15:08:15.234612 (+ 3794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:15.234626 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:15.235080 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:15.235382 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:15.238692 (+ 3310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:15.240272 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:15.242287 (+ 2015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:15.242933 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:15.243061 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:15.248477 (+ 5416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:15.248485 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:15.248501 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:15.248543 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:15.250710 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:15.251528 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:15.251535 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:15.251540 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:15.251687 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:15.252082 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:15.252085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:15.252086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:15.252660 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:15.252877 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:15.253504 (+ 627us) server_negotiation.cc:299] Negotiation successful
0811 15:08:15.253801 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":155,"threads_started":1}
I20260811 15:08:15.295256 30916 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:08:15.296504 30916 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:08:15.307790 30916 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 922d570cf0f94fd58fc91fa911dc9e08: Generated new TSK 0
I20260811 15:08:15.308736 30916 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
I20260811 15:08:15.357851 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:38377
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:39917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:15.493942 30933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:15.494508 30933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:15.494607 30933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:15.501463 30933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:15.501576 30933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:15.501678 30933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:08:15.509550 30933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:39917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:38377
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30933
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:15.511236 30933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:15.512643 30933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:15.526407 30945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:15.526407 30940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:15.526412 30942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:15.526697 30933 server_base.cc:1034] running on GCE node
I20260811 15:08:15.527302 30933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:15.528110 30933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:15.529284 30933 hybrid_clock.cc:648] HybridClock initialized: now 1786460895529264 us; error 41 us; skew 500 ppm
Aug 11 15:08:15 dist-test-slave-1vmg krb5kdc[30762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460895, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:15.532439 30933 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:08:15.533540 30933 webserver.cc:460] Webserver started at http://127.30.8.1:41407/ using document root <none> and password file <none>
I20260811 15:08:15.534207 30933 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:15.534286 30933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:15.534533 30933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:15.536903 30933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "cac9053c63884b58971abcb683a2c3aa"
format_stamp: "Formatted at 2026-08-11 15:08:15 on dist-test-slave-1vmg"
server_key: "08b8a3900c5ff17e0fcb54679dd6c946"
server_key_iv: "8e2c97465a76ef6e89e91d3e89f9116c"
server_key_version: "encryptionkey@0"
I20260811 15:08:15.537513 30933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "cac9053c63884b58971abcb683a2c3aa"
format_stamp: "Formatted at 2026-08-11 15:08:15 on dist-test-slave-1vmg"
server_key: "08b8a3900c5ff17e0fcb54679dd6c946"
server_key_iv: "8e2c97465a76ef6e89e91d3e89f9116c"
server_key_version: "encryptionkey@0"
I20260811 15:08:15.542981 30933 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 15:08:15.545781 30957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:15.547243 30933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:08:15.547392 30933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "cac9053c63884b58971abcb683a2c3aa"
format_stamp: "Formatted at 2026-08-11 15:08:15 on dist-test-slave-1vmg"
server_key: "08b8a3900c5ff17e0fcb54679dd6c946"
server_key_iv: "8e2c97465a76ef6e89e91d3e89f9116c"
server_key_version: "encryptionkey@0"
I20260811 15:08:15.547518 30933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:15.566896 30933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:15.570981 30933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:15.571223 30933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:15.571899 30933 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:15.572858 30933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:15.572911 30933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:15.572968 30933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:15.572996 30933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:15.582445 30933 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:41173
I20260811 15:08:15.582468 31137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:41173 every 8 connection(s)
I20260811 15:08:15.583545 30933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:08:15.588369 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 30933
I20260811 15:08:15.588503 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:08:15.588780 30752 external_mini_cluster.cc:1428] Setting key 229289ba2675db5425e17e4db7fce36c
Aug 11 15:08:15 dist-test-slave-1vmg krb5kdc[30762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460895, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:15.598424 30896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:15.585597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:54967 (local address 127.30.8.62:38377)
0811 15:08:15.585735 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:15.585740 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:15.586841 (+ 1101us) server_negotiation.cc:407] Connection header received
0811 15:08:15.587843 (+ 1002us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:15.587846 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:15.587901 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:15.587966 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:15.589521 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:15.590137 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:15.591440 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:15.591749 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:15.591854 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:15.594711 (+ 2857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:15.594715 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:15.594717 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:15.594743 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:15.596170 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:15.596793 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:15.596795 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:15.596796 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:15.596866 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:15.597242 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:15.597243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:15.597244 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:15.597428 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:15.597500 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:15.598169 (+ 669us) server_negotiation.cc:299] Negotiation successful
0811 15:08:15.598291 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 15:08:15.599828 31143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:15.585969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:38377 (local address 127.30.8.1:54967)
0811 15:08:15.586703 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:15.586747 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:15.587621 (+ 874us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:15.588143 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:15.588152 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:15.588672 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:15.589385 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:15.589403 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:15.590271 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:15.590274 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:15.591317 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:15.591329 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:15.591889 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:15.591892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:15.592046 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:15.592704 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:15.592724 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:15.594524 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:15.596288 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:15.596296 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:15.596315 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:15.596698 (+ 383us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:15.596946 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:15.596949 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:15.596951 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:15.597150 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:15.597521 (+ 371us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:15.597529 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:15.597912 (+ 383us) client_negotiation.cc:769] Sending connection context
0811 15:08:15.598150 (+ 238us) client_negotiation.cc:240] Negotiation successful
0811 15:08:15.598591 (+ 441us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":133,"threads_started":1}
WARNING: no policy specified for kudu/127.30.8.2@KRBTEST.COM; defaulting to no policy
I20260811 15:08:15.601193 31140 heartbeater.cc:347] Connected to a master server at 127.30.8.62:38377
I20260811 15:08:15.601464 31140 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:15.602006 31140 heartbeater.cc:511] Master 127.30.8.62:38377 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.2@KRBTEST.COM" created.
I20260811 15:08:15.603790 30833 ts_manager.cc:194] Registered new tserver with Master: cac9053c63884b58971abcb683a2c3aa (127.30.8.1:41173)
I20260811 15:08:15.604992 30833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:54967
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
I20260811 15:08:15.622522 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:38377
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:39917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:15.754067 31148 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:15.754429 31148 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:15.754500 31148 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:15.758773 31148 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:15.758873 31148 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:15.758991 31148 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:08:15.764904 31148 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:39917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:38377
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31148
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:15.766213 31148 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:15.767407 31148 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:15.778316 31159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:15.778461 31158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:15.778429 31161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:15.778772 31148 server_base.cc:1034] running on GCE node
I20260811 15:08:15.779466 31148 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:15.780490 31148 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:15.781715 31148 hybrid_clock.cc:648] HybridClock initialized: now 1786460895781689 us; error 45 us; skew 500 ppm
Aug 11 15:08:15 dist-test-slave-1vmg krb5kdc[30762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460895, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:15.785022 31148 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:08:15.786164 31148 webserver.cc:460] Webserver started at http://127.30.8.2:33791/ using document root <none> and password file <none>
I20260811 15:08:15.786808 31148 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:15.786883 31148 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:15.787159 31148 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:15.789417 31148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "37116c7e009d499c83ffb3393df30473"
format_stamp: "Formatted at 2026-08-11 15:08:15 on dist-test-slave-1vmg"
server_key: "26e6c3f2a6e767370675effc31910c6f"
server_key_iv: "ca087c81acc93ed4f4b4476495b1f40e"
server_key_version: "encryptionkey@0"
I20260811 15:08:15.789994 31148 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "37116c7e009d499c83ffb3393df30473"
format_stamp: "Formatted at 2026-08-11 15:08:15 on dist-test-slave-1vmg"
server_key: "26e6c3f2a6e767370675effc31910c6f"
server_key_iv: "ca087c81acc93ed4f4b4476495b1f40e"
server_key_version: "encryptionkey@0"
I20260811 15:08:15.794242 31148 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 15:08:15.796937 31174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:15.798322 31148 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 15:08:15.798473 31148 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "37116c7e009d499c83ffb3393df30473"
format_stamp: "Formatted at 2026-08-11 15:08:15 on dist-test-slave-1vmg"
server_key: "26e6c3f2a6e767370675effc31910c6f"
server_key_iv: "ca087c81acc93ed4f4b4476495b1f40e"
server_key_version: "encryptionkey@0"
I20260811 15:08:15.798595 31148 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:15.820858 31148 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:15.825006 31148 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:15.825264 31148 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:15.826098 31148 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:15.827338 31148 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:15.827409 31148 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:15.827476 31148 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:15.827508 31148 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:15.840507 31148 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:41487
I20260811 15:08:15.840554 31368 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:41487 every 8 connection(s)
I20260811 15:08:15.841811 31148 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:08:15.842386 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 31148
I20260811 15:08:15.842506 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:08:15.842794 30752 external_mini_cluster.cc:1428] Setting key 0ccce9d88ccd4d1d2c5fc5d61bbb2645
Aug 11 15:08:15 dist-test-slave-1vmg krb5kdc[30762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460895, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.8.3@KRBTEST.COM; defaulting to no policy
I20260811 15:08:15.857997 30896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:15.844599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:43633 (local address 127.30.8.62:38377)
0811 15:08:15.844734 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:15.844739 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:15.846283 (+ 1544us) server_negotiation.cc:407] Connection header received
0811 15:08:15.847520 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:15.847524 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:15.847587 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:15.847670 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:15.849055 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:15.849784 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:15.851099 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:15.851439 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:15.851593 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:15.854474 (+ 2881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:15.854478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:15.854480 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:15.854509 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:15.855943 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:15.856480 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:15.856482 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:15.856484 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:15.856569 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:15.856855 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:15.856857 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:15.856858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:15.857060 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:15.857134 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:15.857741 (+ 607us) server_negotiation.cc:299] Negotiation successful
0811 15:08:15.857860 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.3@KRBTEST.COM" created.
I20260811 15:08:15.859485 31373 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:15.845130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:38377 (local address 127.30.8.2:43633)
0811 15:08:15.846080 (+ 950us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:15.846148 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:15.847279 (+ 1131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:15.847879 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:15.847887 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:15.848309 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:15.848886 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:15.848901 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:15.849932 (+ 1031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:15.849936 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:15.850890 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:15.850901 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:15.851639 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:15.851642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:15.851803 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:15.852468 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:15.852487 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:15.854237 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:15.856079 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:15.856087 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:15.856104 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:15.856387 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:15.856643 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:15.856645 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:15.856646 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:15.856789 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:15.857136 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:15.857142 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:15.857487 (+ 345us) client_negotiation.cc:769] Sending connection context
0811 15:08:15.857683 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 15:08:15.858005 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":155,"threads_started":1}
I20260811 15:08:15.861472 31370 heartbeater.cc:347] Connected to a master server at 127.30.8.62:38377
I20260811 15:08:15.861833 31370 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:15.862480 31370 heartbeater.cc:511] Master 127.30.8.62:38377 requested a full tablet report, sending...
I20260811 15:08:15.864111 30833 ts_manager.cc:194] Registered new tserver with Master: 37116c7e009d499c83ffb3393df30473 (127.30.8.2:41487)
I20260811 15:08:15.865144 30833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:43633
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
I20260811 15:08:15.879398 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:38377
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:39917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:16.002991 31376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:16.003327 31376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:16.003384 31376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:16.007663 31376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:16.007747 31376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:16.007813 31376 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:08:16.013460 31376 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:39917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:38377
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:16.014647 31376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:16.015729 31376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:16.026109 31383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:16.026109 31384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:16.026109 31387 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:16.026641 31376 server_base.cc:1034] running on GCE node
I20260811 15:08:16.027292 31376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:16.028357 31376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:16.029553 31376 hybrid_clock.cc:648] HybridClock initialized: now 1786460896029497 us; error 71 us; skew 500 ppm
Aug 11 15:08:16 dist-test-slave-1vmg krb5kdc[30762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460896, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:16.032796 31376 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:08:16.033972 31376 webserver.cc:460] Webserver started at http://127.30.8.3:35137/ using document root <none> and password file <none>
I20260811 15:08:16.034588 31376 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:16.034657 31376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:16.034853 31376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:16.037117 31376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "045c5d11fcde4c84b446d411197c5580"
format_stamp: "Formatted at 2026-08-11 15:08:16 on dist-test-slave-1vmg"
server_key: "5739d3e80a2f232d9a3718c5cfd314f8"
server_key_iv: "6e6e2caa5df285f02e9ce027be8d370a"
server_key_version: "encryptionkey@0"
I20260811 15:08:16.037729 31376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "045c5d11fcde4c84b446d411197c5580"
format_stamp: "Formatted at 2026-08-11 15:08:16 on dist-test-slave-1vmg"
server_key: "5739d3e80a2f232d9a3718c5cfd314f8"
server_key_iv: "6e6e2caa5df285f02e9ce027be8d370a"
server_key_version: "encryptionkey@0"
I20260811 15:08:16.042055 31376 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 15:08:16.044782 31398 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:16.046128 31376 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:08:16.046252 31376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "045c5d11fcde4c84b446d411197c5580"
format_stamp: "Formatted at 2026-08-11 15:08:16 on dist-test-slave-1vmg"
server_key: "5739d3e80a2f232d9a3718c5cfd314f8"
server_key_iv: "6e6e2caa5df285f02e9ce027be8d370a"
server_key_version: "encryptionkey@0"
I20260811 15:08:16.046376 31376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:16.062026 31376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:16.065583 31376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:16.065783 31376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:16.066468 31376 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:16.067466 31376 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:16.067518 31376 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:16.067567 31376 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:16.067597 31376 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:16.077188 31376 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:43661
I20260811 15:08:16.077222 31581 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:43661 every 8 connection(s)
I20260811 15:08:16.078235 31376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:08:16.078512 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 31376
I20260811 15:08:16.078668 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:08:16.079030 30752 external_mini_cluster.cc:1428] Setting key 7d13f9c220050907b01d32efe5f93ed2
Aug 11 15:08:16 dist-test-slave-1vmg krb5kdc[30762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460896, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:16.092511 30896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:16.080497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:34447 (local address 127.30.8.62:38377)
0811 15:08:16.080695 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:16.080699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:16.082029 (+ 1330us) server_negotiation.cc:407] Connection header received
0811 15:08:16.083001 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:16.083004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:16.083056 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:16.083126 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:16.084394 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:16.085002 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:16.086125 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:16.086339 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:16.086406 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:16.088967 (+ 2561us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:16.088971 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:16.088973 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:16.088999 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:16.090435 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:16.090973 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:16.090975 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:16.090976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:16.091052 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:16.091377 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:16.091379 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:16.091380 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:16.091581 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:16.091659 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:16.092239 (+ 580us) server_negotiation.cc:299] Negotiation successful
0811 15:08:16.092361 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":109}
I20260811 15:08:16.093739 31584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:16.081044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:38377 (local address 127.30.8.3:34447)
0811 15:08:16.081854 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:16.081914 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:16.082772 (+ 858us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:16.083291 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:16.083298 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:16.083730 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:16.084268 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:16.084284 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:16.085102 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:16.085105 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:16.086026 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:16.086035 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:16.086453 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:16.086456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:16.086621 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:16.087225 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:16.087243 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:16.088783 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:16.090532 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:16.090537 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:16.090558 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:16.090863 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:16.091121 (+ 258us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:16.091123 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:16.091124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:16.091297 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:16.091678 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:16.091684 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:16.091993 (+ 309us) client_negotiation.cc:769] Sending connection context
0811 15:08:16.092196 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 15:08:16.092605 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"thread_start_us":156,"threads_started":1}
I20260811 15:08:16.094854 31582 heartbeater.cc:347] Connected to a master server at 127.30.8.62:38377
I20260811 15:08:16.095125 31582 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:16.095654 31582 heartbeater.cc:511] Master 127.30.8.62:38377 requested a full tablet report, sending...
I20260811 15:08:16.096714 30833 ts_manager.cc:194] Registered new tserver with Master: 045c5d11fcde4c84b446d411197c5580 (127.30.8.3:43661)
I20260811 15:08:16.097314 30833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:34447
I20260811 15:08:16.106145 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 15:08:16 dist-test-slave-1vmg krb5kdc[30762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460896, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 15:08:16 dist-test-slave-1vmg krb5kdc[30762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460896, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.8.1@KRBTEST.COM
I20260811 15:08:16.136369 31591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:16.124863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41618 (local address 127.30.8.1:41173)
0811 15:08:16.125162 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:16.125169 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:16.125216 (+ 47us) server_negotiation.cc:407] Connection header received
0811 15:08:16.125325 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:16.125330 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:16.125540 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:16.125663 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:16.126335 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:16.127191 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:16.128094 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:16.128511 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:16.128598 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:16.131589 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:16.131612 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:16.131637 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:16.131689 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:16.133640 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:16.134262 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:16.134275 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:16.134277 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:16.134477 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:16.134803 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:16.134807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:16.134808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:16.135272 (+ 464us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:16.135532 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:16.135959 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 15:08:16.136172 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":78,"threads_started":1}
W20260811 15:08:16.138123 31002 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:41618
Aug 11 15:08:16 dist-test-slave-1vmg krb5kdc[30762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 15:08:16 dist-test-slave-1vmg krb5kdc[30762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460896, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.1@KRBTEST.COM
I20260811 15:08:16.168752 31591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:16.160830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41630 (local address 127.30.8.1:41173)
0811 15:08:16.160992 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:16.160996 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:16.161071 (+ 75us) server_negotiation.cc:407] Connection header received
0811 15:08:16.161200 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:16.161202 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:16.161252 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:16.161325 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:16.162032 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:16.162545 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:16.163431 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:16.163667 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:16.163747 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:16.166076 (+ 2329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:16.166080 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:16.166082 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:16.166112 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:16.167353 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:16.167742 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:16.167745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:16.167746 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:16.167815 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:16.168003 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:16.168005 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:16.168006 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:16.168181 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:16.168283 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:16.168457 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 15:08:16.168562 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 15:08:16.169823 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 30933
I20260811 15:08:16.187819 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 31148
I20260811 15:08:16.203280 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 31376
I20260811 15:08:16.223273 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 30788
2026-08-11T15:08:16Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2667 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:08:16 dist-test-slave-1vmg krb5kdc[31602](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:08:16 dist-test-slave-1vmg krb5kdc[31602](info): set up 2 sockets
Aug 11 15:08:16 dist-test-slave-1vmg krb5kdc[31602](info): commencing operation
krb5kdc: starting...
W20260811 15:08:17.430714 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.001s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:08:17 dist-test-slave-1vmg krb5kdc[31602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:08:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:08:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:08:17.579552 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:41035
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:44961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:41035
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:17.719658 31630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:17.720031 31630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:17.720078 31630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:17.724481 31630 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:08:17.724567 31630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:17.724588 31630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:17.724604 31630 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:08:17.724625 31630 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:08:17.730549 31630 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:44961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:41035
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:41035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:17.731880 31630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:17.733075 31630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:17.742445 31637 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:17.742441 31645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:17.742456 31638 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:17.742902 31630 server_base.cc:1034] running on GCE node
I20260811 15:08:17.743597 31630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:17.744877 31630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:17.746063 31630 hybrid_clock.cc:648] HybridClock initialized: now 1786460897746044 us; error 41 us; skew 500 ppm
Aug 11 15:08:17 dist-test-slave-1vmg krb5kdc[31602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460897, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:17.749033 31630 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:08:17.750131 31630 webserver.cc:460] Webserver started at http://127.30.8.62:38073/ using document root <none> and password file <none>
I20260811 15:08:17.750766 31630 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:17.750838 31630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:17.751109 31630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:17.753422 31630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "f05b6a61358f4ac4800e96b8824f2637"
format_stamp: "Formatted at 2026-08-11 15:08:17 on dist-test-slave-1vmg"
server_key: "1b564397520c8715351c24a2dd245f32"
server_key_iv: "c5f2bfad53a0a92fc94bb04d0c365c4c"
server_key_version: "encryptionkey@0"
I20260811 15:08:17.754024 31630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "f05b6a61358f4ac4800e96b8824f2637"
format_stamp: "Formatted at 2026-08-11 15:08:17 on dist-test-slave-1vmg"
server_key: "1b564397520c8715351c24a2dd245f32"
server_key_iv: "c5f2bfad53a0a92fc94bb04d0c365c4c"
server_key_version: "encryptionkey@0"
I20260811 15:08:17.758301 31630 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 15:08:17.761137 31658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:17.762516 31630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:08:17.762652 31630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "f05b6a61358f4ac4800e96b8824f2637"
format_stamp: "Formatted at 2026-08-11 15:08:17 on dist-test-slave-1vmg"
server_key: "1b564397520c8715351c24a2dd245f32"
server_key_iv: "c5f2bfad53a0a92fc94bb04d0c365c4c"
server_key_version: "encryptionkey@0"
I20260811 15:08:17.762774 31630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:17.775490 31630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:17.778594 31630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:17.778803 31630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:17.786216 31630 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:41035
I20260811 15:08:17.786222 31752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:41035 every 8 connection(s)
I20260811 15:08:17.787333 31630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:08:17.789842 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 31630
I20260811 15:08:17.789970 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:08:17.790261 30752 external_mini_cluster.cc:1428] Setting key 317c69bd7826ad3f1f360e88f70e7518
I20260811 15:08:17.790865 31753 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:17.798731 31753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637: Bootstrap starting.
Aug 11 15:08:17 dist-test-slave-1vmg krb5kdc[31602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:17.801765 31753 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:17.802748 31753 log.cc:824] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:17.805079 31753 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637: No bootstrap required, opened a new log
I20260811 15:08:17.805982 31760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:17.791779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46524 (local address 127.30.8.62:41035)
0811 15:08:17.792237 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:17.792258 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:17.792305 (+ 47us) server_negotiation.cc:407] Connection header received
0811 15:08:17.793187 (+ 882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:17.793215 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:17.793705 (+ 490us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:17.794130 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:17.794813 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:17.796327 (+ 1514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:17.797175 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:17.797876 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:17.798182 (+ 306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:17.800993 (+ 2811us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:17.801001 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:17.801019 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:17.801061 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:17.803071 (+ 2010us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:17.803516 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:17.803522 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:17.803527 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:17.803654 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:17.803919 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:17.803922 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:17.803923 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:17.804377 (+ 454us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:17.804575 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:17.804871 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 15:08:17.805205 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":130,"threads_started":1}
I20260811 15:08:17.809015 31753 raft_consensus.cc:348] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05b6a61358f4ac4800e96b8824f2637" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 41035 } }
I20260811 15:08:17.809273 31753 raft_consensus.cc:374] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:17.809311 31753 raft_consensus.cc:729] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f05b6a61358f4ac4800e96b8824f2637, State: Initialized, Role: FOLLOWER
I20260811 15:08:17.809764 31753 consensus_queue.cc:260] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05b6a61358f4ac4800e96b8824f2637" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 41035 } }
I20260811 15:08:17.809900 31753 raft_consensus.cc:388] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:08:17.809964 31753 raft_consensus.cc:482] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:08:17.810068 31753 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:17.811120 31753 raft_consensus.cc:504] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05b6a61358f4ac4800e96b8824f2637" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 41035 } }
I20260811 15:08:17.811492 31753 leader_election.cc:302] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f05b6a61358f4ac4800e96b8824f2637; no voters:
I20260811 15:08:17.811839 31753 leader_election.cc:288] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:08:17.811944 31767 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:17.812175 31767 raft_consensus.cc:686] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637 [term 1 LEADER]: Becoming Leader. State: Replica: f05b6a61358f4ac4800e96b8824f2637, State: Running, Role: LEADER
I20260811 15:08:17.812534 31767 consensus_queue.cc:237] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05b6a61358f4ac4800e96b8824f2637" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 41035 } }
I20260811 15:08:17.813172 31753 sys_catalog.cc:564] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:08:17.814217 31768 sys_catalog.cc:455] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f05b6a61358f4ac4800e96b8824f2637" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05b6a61358f4ac4800e96b8824f2637" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 41035 } } }
I20260811 15:08:17.814357 31768 sys_catalog.cc:458] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:17.814778 31769 sys_catalog.cc:455] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f05b6a61358f4ac4800e96b8824f2637. Latest consensus state: current_term: 1 leader_uuid: "f05b6a61358f4ac4800e96b8824f2637" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f05b6a61358f4ac4800e96b8824f2637" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 41035 } } }
I20260811 15:08:17.814883 31769 sys_catalog.cc:458] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:17.815267 31779 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:08:17.818490 31779 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:08:17.824807 31779 catalog_manager.cc:1347] Generated new cluster ID: eb1115c9ef194b3c96d01466acb6cba0
I20260811 15:08:17.824903 31779 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:08:17.839974 31779 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:08:17.841423 31779 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:08:17.854012 31779 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f05b6a61358f4ac4800e96b8824f2637: Generated new TSK 0
I20260811 15:08:17.854856 31779 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
I20260811 15:08:17.894109 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:41035
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:44961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:18.027014 31799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:18.027323 31799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:18.027370 31799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:18.031612 31799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:18.031688 31799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:18.031751 31799 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:08:18.037498 31799 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:44961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:41035
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:18.038719 31799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:18.039837 31799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:18.050235 31809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:18.050293 31799 server_base.cc:1034] running on GCE node
W20260811 15:08:18.050246 31810 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:18.050246 31814 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:18.051035 31799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:18.051760 31799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:18.052944 31799 hybrid_clock.cc:648] HybridClock initialized: now 1786460898052921 us; error 38 us; skew 500 ppm
Aug 11 15:08:18 dist-test-slave-1vmg krb5kdc[31602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460898, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:18.056279 31799 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:08:18.057412 31799 webserver.cc:460] Webserver started at http://127.30.8.1:35027/ using document root <none> and password file <none>
I20260811 15:08:18.058035 31799 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:18.058115 31799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:18.058343 31799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:18.060636 31799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "d77a84fa3bc94f49bc483486d0fc157f"
format_stamp: "Formatted at 2026-08-11 15:08:18 on dist-test-slave-1vmg"
server_key: "f17323538f222745c14d0f1cb390b7e9"
server_key_iv: "4ab2adbc3fe74a55c8b6b871705e6c7d"
server_key_version: "encryptionkey@0"
I20260811 15:08:18.061239 31799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "d77a84fa3bc94f49bc483486d0fc157f"
format_stamp: "Formatted at 2026-08-11 15:08:18 on dist-test-slave-1vmg"
server_key: "f17323538f222745c14d0f1cb390b7e9"
server_key_iv: "4ab2adbc3fe74a55c8b6b871705e6c7d"
server_key_version: "encryptionkey@0"
I20260811 15:08:18.065389 31799 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 15:08:18.068225 31823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:18.069514 31799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 15:08:18.069655 31799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "d77a84fa3bc94f49bc483486d0fc157f"
format_stamp: "Formatted at 2026-08-11 15:08:18 on dist-test-slave-1vmg"
server_key: "f17323538f222745c14d0f1cb390b7e9"
server_key_iv: "4ab2adbc3fe74a55c8b6b871705e6c7d"
server_key_version: "encryptionkey@0"
I20260811 15:08:18.069773 31799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:18.100651 31799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:18.103857 31799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:18.104066 31799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:18.104708 31799 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:18.105625 31799 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:18.105677 31799 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:18.105734 31799 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:18.105762 31799 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:18.115234 31799 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:35329
I20260811 15:08:18.115260 32018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:35329 every 8 connection(s)
I20260811 15:08:18.116254 31799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:08:18.123036 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 31799
I20260811 15:08:18.123163 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:08:18.123497 30752 external_mini_cluster.cc:1428] Setting key db590979a5080d6feb67253699ba9dc3
Aug 11 15:08:18 dist-test-slave-1vmg krb5kdc[31602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460898, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:18.131598 31760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:18.118429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:39871 (local address 127.30.8.62:41035)
0811 15:08:18.118600 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:18.118604 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:18.119500 (+ 896us) server_negotiation.cc:407] Connection header received
0811 15:08:18.120832 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:18.120835 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:18.120886 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:18.120959 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:18.122190 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.122741 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.124383 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.124632 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.124711 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:18.127658 (+ 2947us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:18.127662 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:18.127665 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:18.127698 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:18.129271 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:18.129831 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:18.129835 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:18.129837 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:18.129933 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:18.130257 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:18.130261 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:18.130263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:18.130526 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:18.130635 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:18.131255 (+ 620us) server_negotiation.cc:299] Negotiation successful
0811 15:08:18.131410 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 15:08:18.132400 32023 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:18.118802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:41035 (local address 127.30.8.1:39871)
0811 15:08:18.119337 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:18.119384 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:18.120616 (+ 1232us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:18.121126 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:18.121134 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:18.121519 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:18.122061 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:18.122075 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.122881 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.122885 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:18.124246 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:18.124258 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.124797 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.124802 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:18.124976 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:18.125624 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:18.125647 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:18.127455 (+ 1808us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:18.129401 (+ 1946us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:18.129407 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:18.129425 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:18.129702 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:18.130009 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:18.130011 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:18.130013 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:18.130152 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:18.130626 (+ 474us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:18.130633 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:18.130964 (+ 331us) client_negotiation.cc:769] Sending connection context
0811 15:08:18.131150 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 15:08:18.131459 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":148,"threads_started":1}
I20260811 15:08:18.133689 32019 heartbeater.cc:347] Connected to a master server at 127.30.8.62:41035
I20260811 15:08:18.133944 32019 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:18.134552 32019 heartbeater.cc:511] Master 127.30.8.62:41035 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.8.2@KRBTEST.COM; defaulting to no policy
I20260811 15:08:18.136279 31683 ts_manager.cc:194] Registered new tserver with Master: d77a84fa3bc94f49bc483486d0fc157f (127.30.8.1:35329)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.2@KRBTEST.COM" created.
I20260811 15:08:18.137655 31683 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:39871
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
I20260811 15:08:18.158023 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:41035
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:44961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:18.290731 32029 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:18.291129 32029 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:18.291208 32029 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:18.295485 32029 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:18.295568 32029 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:18.295655 32029 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:08:18.301507 32029 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:44961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:41035
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32029
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:18.302800 32029 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:18.304028 32029 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:18.314606 32037 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:18.314616 32039 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:18.314622 32036 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:18.315146 32029 server_base.cc:1034] running on GCE node
I20260811 15:08:18.315629 32029 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:18.316370 32029 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:18.317665 32029 hybrid_clock.cc:648] HybridClock initialized: now 1786460898317614 us; error 54 us; skew 500 ppm
Aug 11 15:08:18 dist-test-slave-1vmg krb5kdc[31602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460898, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:18.321595 32029 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:08:18.322804 32029 webserver.cc:460] Webserver started at http://127.30.8.2:43881/ using document root <none> and password file <none>
I20260811 15:08:18.323480 32029 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:18.323678 32029 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:18.323961 32029 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:18.326363 32029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "eab0a86bc9ce40fdb1bb84b15b28880a"
format_stamp: "Formatted at 2026-08-11 15:08:18 on dist-test-slave-1vmg"
server_key: "a688d283d17de7c481f087915c46878f"
server_key_iv: "1faf5cf2a5a66d48903b015fb260f9ca"
server_key_version: "encryptionkey@0"
I20260811 15:08:18.327013 32029 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "eab0a86bc9ce40fdb1bb84b15b28880a"
format_stamp: "Formatted at 2026-08-11 15:08:18 on dist-test-slave-1vmg"
server_key: "a688d283d17de7c481f087915c46878f"
server_key_iv: "1faf5cf2a5a66d48903b015fb260f9ca"
server_key_version: "encryptionkey@0"
I20260811 15:08:18.331130 32029 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:08:18.333797 32051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:18.335232 32029 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:08:18.335363 32029 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "eab0a86bc9ce40fdb1bb84b15b28880a"
format_stamp: "Formatted at 2026-08-11 15:08:18 on dist-test-slave-1vmg"
server_key: "a688d283d17de7c481f087915c46878f"
server_key_iv: "1faf5cf2a5a66d48903b015fb260f9ca"
server_key_version: "encryptionkey@0"
I20260811 15:08:18.335481 32029 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:18.372172 32029 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:18.375486 32029 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:18.375691 32029 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:18.376341 32029 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:18.377278 32029 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:18.377328 32029 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:18.377386 32029 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:18.377414 32029 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:18.386703 32029 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:38169
I20260811 15:08:18.386726 32254 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:38169 every 8 connection(s)
I20260811 15:08:18.387732 32029 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
Aug 11 15:08:18 dist-test-slave-1vmg krb5kdc[31602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460898, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:18.397668 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 32029
I20260811 15:08:18.397782 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:08:18.398039 30752 external_mini_cluster.cc:1428] Setting key 8ca2f8a9fb57cdeeabdaadbb766cada5
I20260811 15:08:18.402086 31760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:18.389706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:45585 (local address 127.30.8.62:41035)
0811 15:08:18.389881 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:18.389886 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:18.390816 (+ 930us) server_negotiation.cc:407] Connection header received
0811 15:08:18.392179 (+ 1363us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:18.392182 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:18.392236 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:18.392309 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:18.393563 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.394093 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.395295 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.395516 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.395590 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:18.398330 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:18.398335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:18.398337 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:18.398371 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:18.399934 (+ 1563us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:18.400465 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:18.400468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:18.400470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:18.400569 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:18.400869 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:18.400872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:18.400874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:18.401111 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:18.401206 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:18.401741 (+ 535us) server_negotiation.cc:299] Negotiation successful
0811 15:08:18.401896 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 15:08:18.403088 32258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:18.390149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:41035 (local address 127.30.8.2:45585)
0811 15:08:18.390654 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:18.390701 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:18.391961 (+ 1260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:18.392484 (+ 523us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:18.392491 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:18.392895 (+ 404us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:18.393433 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:18.393448 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.394186 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.394188 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:18.395187 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:18.395198 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.395646 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.395649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:18.395804 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:18.396378 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:18.396395 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:18.398127 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:18.400059 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:18.400066 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:18.400083 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:18.400367 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:18.400651 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:18.400653 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:18.400655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:18.400793 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:18.401196 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:18.401202 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:18.401484 (+ 282us) client_negotiation.cc:769] Sending connection context
0811 15:08:18.401678 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 15:08:18.401978 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":140,"threads_started":1}
I20260811 15:08:18.404276 32255 heartbeater.cc:347] Connected to a master server at 127.30.8.62:41035
I20260811 15:08:18.404537 32255 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:18.405087 32255 heartbeater.cc:511] Master 127.30.8.62:41035 requested a full tablet report, sending...
I20260811 15:08:18.406169 31683 ts_manager.cc:194] Registered new tserver with Master: eab0a86bc9ce40fdb1bb84b15b28880a (127.30.8.2:38169)
I20260811 15:08:18.406775 31683 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:45585
WARNING: no policy specified for kudu/127.30.8.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
I20260811 15:08:18.430840 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:41035
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:44961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:18.572628 32262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:18.572996 32262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:18.573048 32262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:18.577535 32262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:18.577636 32262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:18.577728 32262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:08:18.583992 32262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:44961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:41035
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:18.585536 32262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:18.586882 32262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:18.599140 32276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:18.599146 32272 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:18.599146 32269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:18.599434 32262 server_base.cc:1034] running on GCE node
I20260811 15:08:18.600133 32262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:18.600975 32262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:18.602160 32262 hybrid_clock.cc:648] HybridClock initialized: now 1786460898602143 us; error 52 us; skew 500 ppm
Aug 11 15:08:18 dist-test-slave-1vmg krb5kdc[31602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460898, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:18.605433 32262 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:08:18.606496 32262 webserver.cc:460] Webserver started at http://127.30.8.3:44343/ using document root <none> and password file <none>
I20260811 15:08:18.607218 32262 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:18.607302 32262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:18.607553 32262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:18.609925 32262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "a13f0097e895437da123aeca8c572164"
format_stamp: "Formatted at 2026-08-11 15:08:18 on dist-test-slave-1vmg"
server_key: "6e93a1a8bb486d29ee667ab37d53abb7"
server_key_iv: "b08cd08ecea7dd43b0659bd3ff914858"
server_key_version: "encryptionkey@0"
I20260811 15:08:18.610546 32262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "a13f0097e895437da123aeca8c572164"
format_stamp: "Formatted at 2026-08-11 15:08:18 on dist-test-slave-1vmg"
server_key: "6e93a1a8bb486d29ee667ab37d53abb7"
server_key_iv: "b08cd08ecea7dd43b0659bd3ff914858"
server_key_version: "encryptionkey@0"
I20260811 15:08:18.614982 32262 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.008s sys 0.000s
I20260811 15:08:18.617729 32287 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:18.619772 32262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:08:18.619998 32262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "a13f0097e895437da123aeca8c572164"
format_stamp: "Formatted at 2026-08-11 15:08:18 on dist-test-slave-1vmg"
server_key: "6e93a1a8bb486d29ee667ab37d53abb7"
server_key_iv: "b08cd08ecea7dd43b0659bd3ff914858"
server_key_version: "encryptionkey@0"
I20260811 15:08:18.620167 32262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:18.633437 32262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:18.637084 32262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:18.637300 32262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:18.637982 32262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:18.639070 32262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:18.639123 32262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:18.639183 32262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:18.639209 32262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:18.648726 32262 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:45885
I20260811 15:08:18.648746 32474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:45885 every 8 connection(s)
I20260811 15:08:18.649761 32262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:08:18.651479 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 32262
I20260811 15:08:18.651620 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:08:18.651940 30752 external_mini_cluster.cc:1428] Setting key 44b98b8291624703c44c50995779819d
Aug 11 15:08:18 dist-test-slave-1vmg krb5kdc[31602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460898, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:18.669903 31760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:18.652214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:53887 (local address 127.30.8.62:41035)
0811 15:08:18.652346 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:18.652350 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:18.653243 (+ 893us) server_negotiation.cc:407] Connection header received
0811 15:08:18.654819 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:18.654822 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:18.654882 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:18.654986 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:18.656294 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.656827 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.658020 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.658251 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.658344 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:18.665629 (+ 7285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:18.665634 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:18.665637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:18.665672 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:18.667516 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:18.668209 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:18.668213 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:18.668215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:18.668306 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:18.668675 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:18.668679 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:18.668680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:18.668897 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:18.668993 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:18.669599 (+ 606us) server_negotiation.cc:299] Negotiation successful
0811 15:08:18.669738 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 15:08:18.670892 32481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:18.652465 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:41035 (local address 127.30.8.3:53887)
0811 15:08:18.653088 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:18.653149 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:18.654596 (+ 1447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:18.655185 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:18.655193 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:18.655598 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:18.656146 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:18.656161 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.656951 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.656955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:18.657893 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:18.657904 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.658404 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.658408 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:18.658598 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:18.663273 (+ 4675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:18.663313 (+ 40us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:18.665396 (+ 2083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:18.667684 (+ 2288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:18.667693 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:18.667714 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:18.668095 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:18.668394 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:18.668397 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:18.668399 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:18.668576 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:18.669017 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:18.669026 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:18.669334 (+ 308us) client_negotiation.cc:769] Sending connection context
0811 15:08:18.669537 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 15:08:18.669956 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":176,"threads_started":1}
I20260811 15:08:18.672138 32479 heartbeater.cc:347] Connected to a master server at 127.30.8.62:41035
I20260811 15:08:18.672380 32479 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:18.672922 32479 heartbeater.cc:511] Master 127.30.8.62:41035 requested a full tablet report, sending...
I20260811 15:08:18.674121 31683 ts_manager.cc:194] Registered new tserver with Master: a13f0097e895437da123aeca8c572164 (127.30.8.3:45885)
I20260811 15:08:18.674695 31683 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:53887
I20260811 15:08:18.679490 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 15:08:18.691490 31760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:18.683646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46532 (local address 127.30.8.62:41035)
0811 15:08:18.683786 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:18.683791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:18.684066 (+ 275us) server_negotiation.cc:407] Connection header received
0811 15:08:18.684236 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:18.684240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:18.684286 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:18.684358 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:18.685109 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.685639 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.686488 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.686713 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.686801 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:18.688411 (+ 1610us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:18.688415 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:18.688417 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:18.688445 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:18.689768 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:18.690190 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:18.690193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:18.690194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:18.690271 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:18.690527 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:18.690530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:18.690532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:18.690777 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:18.690904 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:18.691198 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 15:08:18.691323 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 15:08:18.696849 31683 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46532:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:08:18.700145 31683 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:08:18.715359 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:18.709936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32852 (local address 127.30.8.2:38169)
0811 15:08:18.710590 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:18.710706 (+ 116us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:18.710735 (+ 29us) server_negotiation.cc:407] Connection header received
0811 15:08:18.710814 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:18.710819 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:18.711047 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:18.711192 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:18.711409 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.712139 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.713960 (+ 1821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.714629 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.714729 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:18.714897 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:18.715094 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 15:08:18.715198 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":136,"threads_started":1}
I20260811 15:08:18.715355 32501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:18.709174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:38169 (local address 127.0.0.1:32852)
0811 15:08:18.710105 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:18.710118 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:18.710212 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:18.711171 (+ 959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:18.711174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:18.711192 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:18.711299 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:18.711306 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.712260 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.712264 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:18.713810 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:18.713849 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.714741 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.714744 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:18.714865 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:18.714906 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 15:08:18.715048 (+ 142us) client_negotiation.cc:240] Negotiation successful
0811 15:08:18.715191 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":867,"mutex_wait_us":49,"thread_start_us":113,"threads_started":1}
I20260811 15:08:18.715903 32500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:18.709184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41052 (local address 127.30.8.1:35329)
0811 15:08:18.709534 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:18.709542 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:18.710187 (+ 645us) server_negotiation.cc:407] Connection header received
0811 15:08:18.710280 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:18.710285 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:18.710510 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:18.710674 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:18.711200 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.712369 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.713966 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.714755 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.714864 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:18.715633 (+ 769us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:18.715682 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 15:08:18.715765 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":130,"threads_started":1}
I20260811 15:08:18.715900 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:18.710335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34088 (local address 127.30.8.3:45885)
0811 15:08:18.710773 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:18.710785 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:18.710805 (+ 20us) server_negotiation.cc:407] Connection header received
0811 15:08:18.710875 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:18.710880 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:18.711092 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:18.711509 (+ 417us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:18.711663 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.712662 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.714210 (+ 1548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.715049 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.715452 (+ 403us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:18.715594 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:18.715663 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 15:08:18.715756 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":150,"threads_started":1}
I20260811 15:08:18.716149 32497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:18.709041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:35329 (local address 127.0.0.1:41052)
0811 15:08:18.709700 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:18.709754 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:18.710027 (+ 273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:18.710858 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:18.710865 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:18.710888 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:18.711056 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:18.711064 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.712482 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.712485 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:18.713810 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:18.713822 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.715014 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.715018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:18.715104 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:18.715148 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 15:08:18.715206 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 15:08:18.715283 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"mutex_wait_us":61,"thread_start_us":120,"threads_started":1}
I20260811 15:08:18.716192 32498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:18.709866 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:45885 (local address 127.0.0.1:34088)
0811 15:08:18.710143 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:18.710154 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:18.710217 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:18.711255 (+ 1038us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:18.711258 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:18.711278 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:18.711369 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:18.711376 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.712785 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.712788 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:18.714068 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:18.714079 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.715173 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.715176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:18.715254 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:18.715274 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 15:08:18.715314 (+ 40us) client_negotiation.cc:240] Negotiation successful
0811 15:08:18.715373 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"mutex_wait_us":171,"thread_start_us":107,"threads_started":1}
I20260811 15:08:18.718945 31899 tablet_service.cc:1467] Processing CreateTablet for tablet 1c24661c139141d3a113a4cd1f7308f0 (DEFAULT_TABLE table=test-table [id=9a317b770a3d41a08cc1800319e3f138]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:18.718966 32133 tablet_service.cc:1467] Processing CreateTablet for tablet 1c24661c139141d3a113a4cd1f7308f0 (DEFAULT_TABLE table=test-table [id=9a317b770a3d41a08cc1800319e3f138]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:18.719257 32368 tablet_service.cc:1467] Processing CreateTablet for tablet 1c24661c139141d3a113a4cd1f7308f0 (DEFAULT_TABLE table=test-table [id=9a317b770a3d41a08cc1800319e3f138]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:18.720284 32133 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c24661c139141d3a113a4cd1f7308f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:18.720284 31899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c24661c139141d3a113a4cd1f7308f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:18.720803 32368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1c24661c139141d3a113a4cd1f7308f0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:18.729031 32508 tablet_bootstrap.cc:492] T 1c24661c139141d3a113a4cd1f7308f0 P a13f0097e895437da123aeca8c572164: Bootstrap starting.
I20260811 15:08:18.729765 32506 tablet_bootstrap.cc:492] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f: Bootstrap starting.
I20260811 15:08:18.730619 32509 tablet_bootstrap.cc:492] T 1c24661c139141d3a113a4cd1f7308f0 P eab0a86bc9ce40fdb1bb84b15b28880a: Bootstrap starting.
I20260811 15:08:18.733016 32506 tablet_bootstrap.cc:654] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:18.733162 32509 tablet_bootstrap.cc:654] T 1c24661c139141d3a113a4cd1f7308f0 P eab0a86bc9ce40fdb1bb84b15b28880a: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:18.733209 32508 tablet_bootstrap.cc:654] T 1c24661c139141d3a113a4cd1f7308f0 P a13f0097e895437da123aeca8c572164: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:18.734193 32506 log.cc:824] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:18.734606 32509 log.cc:824] T 1c24661c139141d3a113a4cd1f7308f0 P eab0a86bc9ce40fdb1bb84b15b28880a: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:18.734756 32508 log.cc:824] T 1c24661c139141d3a113a4cd1f7308f0 P a13f0097e895437da123aeca8c572164: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:18.736943 32506 tablet_bootstrap.cc:492] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f: No bootstrap required, opened a new log
I20260811 15:08:18.736944 32509 tablet_bootstrap.cc:492] T 1c24661c139141d3a113a4cd1f7308f0 P eab0a86bc9ce40fdb1bb84b15b28880a: No bootstrap required, opened a new log
I20260811 15:08:18.737181 32509 ts_tablet_manager.cc:1397] T 1c24661c139141d3a113a4cd1f7308f0 P eab0a86bc9ce40fdb1bb84b15b28880a: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 15:08:18.737192 32506 ts_tablet_manager.cc:1397] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 15:08:18.740998 32508 tablet_bootstrap.cc:492] T 1c24661c139141d3a113a4cd1f7308f0 P a13f0097e895437da123aeca8c572164: No bootstrap required, opened a new log
I20260811 15:08:18.740855 32509 raft_consensus.cc:348] T 1c24661c139141d3a113a4cd1f7308f0 P eab0a86bc9ce40fdb1bb84b15b28880a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d77a84fa3bc94f49bc483486d0fc157f" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35329 } } peers { permanent_uuid: "eab0a86bc9ce40fdb1bb84b15b28880a" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38169 } } peers { permanent_uuid: "a13f0097e895437da123aeca8c572164" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45885 } }
I20260811 15:08:18.741191 32509 raft_consensus.cc:374] T 1c24661c139141d3a113a4cd1f7308f0 P eab0a86bc9ce40fdb1bb84b15b28880a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:18.741256 32509 raft_consensus.cc:729] T 1c24661c139141d3a113a4cd1f7308f0 P eab0a86bc9ce40fdb1bb84b15b28880a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eab0a86bc9ce40fdb1bb84b15b28880a, State: Initialized, Role: FOLLOWER
I20260811 15:08:18.741501 32508 ts_tablet_manager.cc:1397] T 1c24661c139141d3a113a4cd1f7308f0 P a13f0097e895437da123aeca8c572164: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.000s
I20260811 15:08:18.741812 32509 consensus_queue.cc:260] T 1c24661c139141d3a113a4cd1f7308f0 P eab0a86bc9ce40fdb1bb84b15b28880a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d77a84fa3bc94f49bc483486d0fc157f" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35329 } } peers { permanent_uuid: "eab0a86bc9ce40fdb1bb84b15b28880a" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38169 } } peers { permanent_uuid: "a13f0097e895437da123aeca8c572164" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45885 } }
I20260811 15:08:18.741875 32506 raft_consensus.cc:348] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d77a84fa3bc94f49bc483486d0fc157f" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35329 } } peers { permanent_uuid: "eab0a86bc9ce40fdb1bb84b15b28880a" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38169 } } peers { permanent_uuid: "a13f0097e895437da123aeca8c572164" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45885 } }
I20260811 15:08:18.742183 32506 raft_consensus.cc:374] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:18.742254 32506 raft_consensus.cc:729] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d77a84fa3bc94f49bc483486d0fc157f, State: Initialized, Role: FOLLOWER
I20260811 15:08:18.742683 32255 heartbeater.cc:503] Master 127.30.8.62:41035 was elected leader, sending a full tablet report...
I20260811 15:08:18.742849 32506 consensus_queue.cc:260] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d77a84fa3bc94f49bc483486d0fc157f" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35329 } } peers { permanent_uuid: "eab0a86bc9ce40fdb1bb84b15b28880a" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38169 } } peers { permanent_uuid: "a13f0097e895437da123aeca8c572164" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45885 } }
I20260811 15:08:18.743901 32019 heartbeater.cc:503] Master 127.30.8.62:41035 was elected leader, sending a full tablet report...
I20260811 15:08:18.744125 32506 ts_tablet_manager.cc:1428] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 15:08:18.744736 32509 ts_tablet_manager.cc:1428] T 1c24661c139141d3a113a4cd1f7308f0 P eab0a86bc9ce40fdb1bb84b15b28880a: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 15:08:18.745870 32508 raft_consensus.cc:348] T 1c24661c139141d3a113a4cd1f7308f0 P a13f0097e895437da123aeca8c572164 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d77a84fa3bc94f49bc483486d0fc157f" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35329 } } peers { permanent_uuid: "eab0a86bc9ce40fdb1bb84b15b28880a" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38169 } } peers { permanent_uuid: "a13f0097e895437da123aeca8c572164" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45885 } }
I20260811 15:08:18.746223 32508 raft_consensus.cc:374] T 1c24661c139141d3a113a4cd1f7308f0 P a13f0097e895437da123aeca8c572164 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:18.746294 32508 raft_consensus.cc:729] T 1c24661c139141d3a113a4cd1f7308f0 P a13f0097e895437da123aeca8c572164 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a13f0097e895437da123aeca8c572164, State: Initialized, Role: FOLLOWER
I20260811 15:08:18.746896 32508 consensus_queue.cc:260] T 1c24661c139141d3a113a4cd1f7308f0 P a13f0097e895437da123aeca8c572164 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d77a84fa3bc94f49bc483486d0fc157f" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35329 } } peers { permanent_uuid: "eab0a86bc9ce40fdb1bb84b15b28880a" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38169 } } peers { permanent_uuid: "a13f0097e895437da123aeca8c572164" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45885 } }
I20260811 15:08:18.747789 32479 heartbeater.cc:503] Master 127.30.8.62:41035 was elected leader, sending a full tablet report...
I20260811 15:08:18.748277 32508 ts_tablet_manager.cc:1428] T 1c24661c139141d3a113a4cd1f7308f0 P a13f0097e895437da123aeca8c572164: Time spent starting tablet: real 0.007s user 0.002s sys 0.004s
I20260811 15:08:18.810317 32514 raft_consensus.cc:482] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:08:18.810563 32514 raft_consensus.cc:504] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d77a84fa3bc94f49bc483486d0fc157f" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35329 } } peers { permanent_uuid: "eab0a86bc9ce40fdb1bb84b15b28880a" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38169 } } peers { permanent_uuid: "a13f0097e895437da123aeca8c572164" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45885 } }
I20260811 15:08:18.811903 32514 leader_election.cc:288] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eab0a86bc9ce40fdb1bb84b15b28880a (127.30.8.2:38169), a13f0097e895437da123aeca8c572164 (127.30.8.3:45885)
I20260811 15:08:18.816042 32504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:18.812217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:39319 (local address 127.30.8.2:38169)
0811 15:08:18.812357 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:18.812362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:18.812642 (+ 280us) server_negotiation.cc:407] Connection header received
0811 15:08:18.812783 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:18.812787 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:18.812835 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:18.812895 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:18.813303 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.813872 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.815032 (+ 1160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.815495 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.815590 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:18.815654 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:18.815854 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 15:08:18.815924 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 15:08:18.816039 32505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:18.812333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:41979 (local address 127.30.8.3:45885)
0811 15:08:18.812465 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:18.812469 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:18.812713 (+ 244us) server_negotiation.cc:407] Connection header received
0811 15:08:18.812828 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:18.812831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:18.812874 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:18.812936 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:18.813284 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.813820 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.814995 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.815470 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.815571 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:18.815642 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:18.815839 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 15:08:18.815907 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 15:08:18.816066 32516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:18.812212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:45885 (local address 127.30.8.1:41979)
0811 15:08:18.812620 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:18.812631 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:18.812738 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:18.812955 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:18.812958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:18.812973 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:18.813177 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:18.813183 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.813922 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.813925 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:18.814855 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:18.814866 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.815596 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.815600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:18.815698 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:18.815740 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 15:08:18.815807 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 15:08:18.815887 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":32,"thread_start_us":131,"threads_started":1}
I20260811 15:08:18.816072 32517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:18.812148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:38169 (local address 127.30.8.1:39319)
0811 15:08:18.812549 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:18.812561 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:18.812679 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:18.812916 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:18.812918 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:18.812933 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:18.813166 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:18.813172 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.813972 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.813975 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:18.814891 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:18.814901 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.815607 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.815610 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:18.815747 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:18.815765 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 15:08:18.815828 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 15:08:18.815903 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":42,"spinlock_wait_cycles":12672,"thread_start_us":149,"threads_started":1}
I20260811 15:08:18.816890 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c24661c139141d3a113a4cd1f7308f0" candidate_uuid: "d77a84fa3bc94f49bc483486d0fc157f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eab0a86bc9ce40fdb1bb84b15b28880a" is_pre_election: true
I20260811 15:08:18.816891 32400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c24661c139141d3a113a4cd1f7308f0" candidate_uuid: "d77a84fa3bc94f49bc483486d0fc157f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a13f0097e895437da123aeca8c572164" is_pre_election: true
I20260811 15:08:18.817263 32400 raft_consensus.cc:2454] T 1c24661c139141d3a113a4cd1f7308f0 P a13f0097e895437da123aeca8c572164 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d77a84fa3bc94f49bc483486d0fc157f in term 0.
I20260811 15:08:18.817263 32170 raft_consensus.cc:2454] T 1c24661c139141d3a113a4cd1f7308f0 P eab0a86bc9ce40fdb1bb84b15b28880a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d77a84fa3bc94f49bc483486d0fc157f in term 0.
I20260811 15:08:18.817807 31826 leader_election.cc:302] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a13f0097e895437da123aeca8c572164, d77a84fa3bc94f49bc483486d0fc157f; no voters:
I20260811 15:08:18.818109 32514 raft_consensus.cc:2781] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:08:18.818199 32514 raft_consensus.cc:482] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:08:18.818253 32514 raft_consensus.cc:3037] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:18.819558 32514 raft_consensus.cc:504] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d77a84fa3bc94f49bc483486d0fc157f" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35329 } } peers { permanent_uuid: "eab0a86bc9ce40fdb1bb84b15b28880a" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38169 } } peers { permanent_uuid: "a13f0097e895437da123aeca8c572164" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45885 } }
I20260811 15:08:18.819950 32514 leader_election.cc:288] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f [CANDIDATE]: Term 1 election: Requested vote from peers eab0a86bc9ce40fdb1bb84b15b28880a (127.30.8.2:38169), a13f0097e895437da123aeca8c572164 (127.30.8.3:45885)
I20260811 15:08:18.820387 32170 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c24661c139141d3a113a4cd1f7308f0" candidate_uuid: "d77a84fa3bc94f49bc483486d0fc157f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eab0a86bc9ce40fdb1bb84b15b28880a"
I20260811 15:08:18.820387 32400 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1c24661c139141d3a113a4cd1f7308f0" candidate_uuid: "d77a84fa3bc94f49bc483486d0fc157f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a13f0097e895437da123aeca8c572164"
I20260811 15:08:18.820528 32400 raft_consensus.cc:3037] T 1c24661c139141d3a113a4cd1f7308f0 P a13f0097e895437da123aeca8c572164 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:18.820528 32170 raft_consensus.cc:3037] T 1c24661c139141d3a113a4cd1f7308f0 P eab0a86bc9ce40fdb1bb84b15b28880a [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:18.821962 32400 raft_consensus.cc:2454] T 1c24661c139141d3a113a4cd1f7308f0 P a13f0097e895437da123aeca8c572164 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d77a84fa3bc94f49bc483486d0fc157f in term 1.
I20260811 15:08:18.821961 32170 raft_consensus.cc:2454] T 1c24661c139141d3a113a4cd1f7308f0 P eab0a86bc9ce40fdb1bb84b15b28880a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d77a84fa3bc94f49bc483486d0fc157f in term 1.
I20260811 15:08:18.822395 31824 leader_election.cc:302] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d77a84fa3bc94f49bc483486d0fc157f, eab0a86bc9ce40fdb1bb84b15b28880a; no voters:
I20260811 15:08:18.822603 32514 raft_consensus.cc:2781] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:18.822855 32514 raft_consensus.cc:686] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f [term 1 LEADER]: Becoming Leader. State: Replica: d77a84fa3bc94f49bc483486d0fc157f, State: Running, Role: LEADER
I20260811 15:08:18.823257 32514 consensus_queue.cc:237] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d77a84fa3bc94f49bc483486d0fc157f" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35329 } } peers { permanent_uuid: "eab0a86bc9ce40fdb1bb84b15b28880a" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38169 } } peers { permanent_uuid: "a13f0097e895437da123aeca8c572164" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45885 } }
I20260811 15:08:18.826740 31683 catalog_manager.cc:5495] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f reported cstate change: term changed from 0 to 1, leader changed from <none> to d77a84fa3bc94f49bc483486d0fc157f (127.30.8.1). New cstate: current_term: 1 leader_uuid: "d77a84fa3bc94f49bc483486d0fc157f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d77a84fa3bc94f49bc483486d0fc157f" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eab0a86bc9ce40fdb1bb84b15b28880a" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38169 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a13f0097e895437da123aeca8c572164" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45885 } health_report { overall_health: UNKNOWN } } }
Aug 11 15:08:18 dist-test-slave-1vmg krb5kdc[31602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460897, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.1@KRBTEST.COM
I20260811 15:08:18.843891 32500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:18.835016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41062 (local address 127.30.8.1:35329)
0811 15:08:18.835167 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:18.835171 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:18.835255 (+ 84us) server_negotiation.cc:407] Connection header received
0811 15:08:18.835364 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:18.835366 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:18.835408 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:18.835476 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:18.836197 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.836712 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.837473 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.837680 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.837746 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:18.840321 (+ 2575us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:18.840334 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:18.840355 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:18.840381 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:18.841888 (+ 1507us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:18.842292 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:18.842297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:18.842298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:18.842378 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:18.842648 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:18.842652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:18.842654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:18.843066 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:18.843302 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:18.843560 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 15:08:18.843712 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 15:08:18 dist-test-slave-1vmg krb5kdc[31602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460898, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
W20260811 15:08:18.867621 32020 tablet.cc:2369] T 1c24661c139141d3a113a4cd1f7308f0 P d77a84fa3bc94f49bc483486d0fc157f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 15:08:18 dist-test-slave-1vmg krb5kdc[31602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460898, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.8.1@KRBTEST.COM
I20260811 15:08:18.873039 32500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:18.864978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41078 (local address 127.30.8.1:35329)
0811 15:08:18.865170 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:18.865174 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:18.865237 (+ 63us) server_negotiation.cc:407] Connection header received
0811 15:08:18.865376 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:18.865379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:18.865426 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:18.865488 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:18.866172 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.866745 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.867609 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.867909 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.868014 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:18.870358 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:18.870362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:18.870364 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:18.870389 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:18.871632 (+ 1243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:18.871993 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:18.871996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:18.871997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:18.872071 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:18.872254 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:18.872256 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:18.872257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:18.872422 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:18.872538 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:18.872760 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 15:08:18.872866 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
Aug 11 15:08:18 dist-test-slave-1vmg krb5kdc[31602](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260811 15:08:18.889031 32256 tablet.cc:2369] T 1c24661c139141d3a113a4cd1f7308f0 P eab0a86bc9ce40fdb1bb84b15b28880a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 15:08:18 dist-test-slave-1vmg krb5kdc[31602](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460898, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.1@KRBTEST.COM
I20260811 15:08:18.900961 32500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:18.892702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41084 (local address 127.30.8.1:35329)
0811 15:08:18.892865 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:18.892871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:18.892978 (+ 107us) server_negotiation.cc:407] Connection header received
0811 15:08:18.893102 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:18.893104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:18.893173 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:18.893244 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:18.894071 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.894583 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.895429 (+ 846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:18.895651 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:18.895730 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:18.898137 (+ 2407us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:18.898141 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:18.898143 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:18.898178 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:18.899541 (+ 1363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:18.899916 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:18.899918 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:18.899920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:18.899991 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:18.900192 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:18.900195 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:18.900196 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:18.900365 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:18.900482 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:18.900653 (+ 171us) server_negotiation.cc:299] Negotiation successful
0811 15:08:18.900790 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 15:08:18.901014 32480 tablet.cc:2369] T 1c24661c139141d3a113a4cd1f7308f0 P a13f0097e895437da123aeca8c572164: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 15:08:18.902518 31868 tablet_service.cc:2908] Rejecting scan request for tablet 1c24661c139141d3a113a4cd1f7308f0: Uninitialized: clean time has not yet been initialized
I20260811 15:08:18.904803 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 31799
I20260811 15:08:18.927942 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 32029
I20260811 15:08:18.945681 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 32262
I20260811 15:08:18.962980 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 31630
2026-08-11T15:08:18Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2729 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:08:19 dist-test-slave-1vmg krb5kdc[32542](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:08:19 dist-test-slave-1vmg krb5kdc[32542](info): set up 2 sockets
Aug 11 15:08:19 dist-test-slave-1vmg krb5kdc[32542](info): commencing operation
krb5kdc: starting...
W20260811 15:08:20.137739 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:08:20 dist-test-slave-1vmg krb5kdc[32542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:08:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:08:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:08:20.264225 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:43365
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:41899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:43365
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:20.387126 32563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:20.387462 32563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:20.387507 32563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:20.391621 32563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:08:20.391690 32563 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 15:08:20.391706 32563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:20.391723 32563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:20.391749 32563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:08:20.391778 32563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:08:20.397382 32563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:41899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:43365
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:43365
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:20.398619 32563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:20.399758 32563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:20.408444 32576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:20.408452 32572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:20.408458 32574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:20.408710 32563 server_base.cc:1034] running on GCE node
I20260811 15:08:20.409408 32563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:20.410580 32563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:20.411741 32563 hybrid_clock.cc:648] HybridClock initialized: now 1786460900411725 us; error 32 us; skew 500 ppm
Aug 11 15:08:20 dist-test-slave-1vmg krb5kdc[32542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460900, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:20.414448 32563 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:08:20.415503 32563 webserver.cc:460] Webserver started at http://127.30.8.62:45737/ using document root <none> and password file <none>
I20260811 15:08:20.416083 32563 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:20.416149 32563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:20.416350 32563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:20.418510 32563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "a19dd1a8b19947ed957a34d45d46c83c"
format_stamp: "Formatted at 2026-08-11 15:08:20 on dist-test-slave-1vmg"
server_key: "03e0ad92c993c0a57dd9fa467bbe79f1"
server_key_iv: "8adfb360fa3bbb902874dcd46e0edd31"
server_key_version: "encryptionkey@0"
I20260811 15:08:20.419155 32563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "a19dd1a8b19947ed957a34d45d46c83c"
format_stamp: "Formatted at 2026-08-11 15:08:20 on dist-test-slave-1vmg"
server_key: "03e0ad92c993c0a57dd9fa467bbe79f1"
server_key_iv: "8adfb360fa3bbb902874dcd46e0edd31"
server_key_version: "encryptionkey@0"
I20260811 15:08:20.423208 32563 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:08:20.425843 32590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:20.427250 32563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:08:20.427385 32563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "a19dd1a8b19947ed957a34d45d46c83c"
format_stamp: "Formatted at 2026-08-11 15:08:20 on dist-test-slave-1vmg"
server_key: "03e0ad92c993c0a57dd9fa467bbe79f1"
server_key_iv: "8adfb360fa3bbb902874dcd46e0edd31"
server_key_version: "encryptionkey@0"
I20260811 15:08:20.427498 32563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:20.440040 32563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:20.443324 32563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:20.443543 32563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:20.452549 32563 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:43365
I20260811 15:08:20.452582 32690 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:43365 every 8 connection(s)
I20260811 15:08:20.453609 32563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:08:20.458127 32691 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:20.461287 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 32563
I20260811 15:08:20.461400 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:08:20.461678 30752 external_mini_cluster.cc:1428] Setting key 29ca87b8e3b9ea8f57f3d06c519453db
I20260811 15:08:20.467264 32691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c: Bootstrap starting.
I20260811 15:08:20.470247 32691 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:20.471330 32691 log.cc:824] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c: Log is configured to *not* fsync() on all Append() calls
Aug 11 15:08:20 dist-test-slave-1vmg krb5kdc[32542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460900, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:20.474105 32691 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c: No bootstrap required, opened a new log
I20260811 15:08:20.478384 32691 raft_consensus.cc:348] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19dd1a8b19947ed957a34d45d46c83c" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43365 } }
I20260811 15:08:20.478724 32691 raft_consensus.cc:374] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:20.478783 32691 raft_consensus.cc:729] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a19dd1a8b19947ed957a34d45d46c83c, State: Initialized, Role: FOLLOWER
I20260811 15:08:20.479351 32691 consensus_queue.cc:260] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19dd1a8b19947ed957a34d45d46c83c" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43365 } }
I20260811 15:08:20.479553 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:20.463391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49648 (local address 127.30.8.62:43365)
0811 15:08:20.463903 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:20.463923 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:20.463968 (+ 45us) server_negotiation.cc:407] Connection header received
0811 15:08:20.464801 (+ 833us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:20.464824 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:20.465218 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:20.465556 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:20.466617 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:20.467743 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:20.468929 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:20.469437 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:20.469587 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:20.472814 (+ 3227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:20.472821 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:20.472841 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:20.472873 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:20.476161 (+ 3288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:20.476650 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:20.476658 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:20.476665 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:20.476829 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:20.477070 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:20.477074 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:20.477076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:20.477673 (+ 597us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:20.477898 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:20.478117 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 15:08:20.478451 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":125,"threads_started":1}
I20260811 15:08:20.479537 32691 raft_consensus.cc:388] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:08:20.479672 32691 raft_consensus.cc:482] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:08:20.479787 32691 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:20.480867 32691 raft_consensus.cc:504] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19dd1a8b19947ed957a34d45d46c83c" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43365 } }
I20260811 15:08:20.481257 32691 leader_election.cc:302] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a19dd1a8b19947ed957a34d45d46c83c; no voters:
I20260811 15:08:20.481652 32691 leader_election.cc:288] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:08:20.481760 32711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:20.481989 32711 raft_consensus.cc:686] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c [term 1 LEADER]: Becoming Leader. State: Replica: a19dd1a8b19947ed957a34d45d46c83c, State: Running, Role: LEADER
I20260811 15:08:20.482316 32711 consensus_queue.cc:237] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19dd1a8b19947ed957a34d45d46c83c" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43365 } }
I20260811 15:08:20.482980 32691 sys_catalog.cc:564] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:08:20.484076 32714 sys_catalog.cc:455] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c [sys.catalog]: SysCatalogTable state changed. Reason: New leader a19dd1a8b19947ed957a34d45d46c83c. Latest consensus state: current_term: 1 leader_uuid: "a19dd1a8b19947ed957a34d45d46c83c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19dd1a8b19947ed957a34d45d46c83c" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43365 } } }
I20260811 15:08:20.484213 32714 sys_catalog.cc:458] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:20.484506 32713 sys_catalog.cc:455] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a19dd1a8b19947ed957a34d45d46c83c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19dd1a8b19947ed957a34d45d46c83c" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43365 } } }
I20260811 15:08:20.484604 32713 sys_catalog.cc:458] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:20.484591 32726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:08:20.487739 32726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:08:20.493446 32726 catalog_manager.cc:1347] Generated new cluster ID: 1f83e6aed6db4a0e92f5319520b505ef
I20260811 15:08:20.493556 32726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:08:20.523213 32726 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:08:20.524310 32726 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:08:20.531947 32726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c: Generated new TSK 0
I20260811 15:08:20.532727 32726 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
I20260811 15:08:20.570214 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:43365
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:41899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:20.695111 32747 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:20.695442 32747 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:20.695493 32747 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:20.699878 32747 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:20.699978 32747 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:20.700030 32747 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 15:08:20.700083 32747 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:08:20.705878 32747 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:41899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.8.62:43365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32747
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:20.707252 32747 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:20.708453 32747 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:20.719275 32755 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:20.719313 32758 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:20.719513 32747 server_base.cc:1034] running on GCE node
W20260811 15:08:20.719516 32756 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:20.720055 32747 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:20.720781 32747 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:20.721987 32747 hybrid_clock.cc:648] HybridClock initialized: now 1786460900721970 us; error 39 us; skew 500 ppm
Aug 11 15:08:20 dist-test-slave-1vmg krb5kdc[32542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460900, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:20.725096 32747 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:08:20.726136 32747 webserver.cc:460] Webserver started at http://127.30.8.1:37115/ using document root <none> and password file <none>
I20260811 15:08:20.726745 32747 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:20.726816 32747 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:20.727064 32747 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:20.729346 32747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "7f1c581731e14b9fa50634b9a50957e1"
format_stamp: "Formatted at 2026-08-11 15:08:20 on dist-test-slave-1vmg"
server_key: "50e46bf0c5ee8ae4c174ad75790e8b5b"
server_key_iv: "b6a2b8723d74707e90adb19e77e2e570"
server_key_version: "encryptionkey@0"
I20260811 15:08:20.729946 32747 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "7f1c581731e14b9fa50634b9a50957e1"
format_stamp: "Formatted at 2026-08-11 15:08:20 on dist-test-slave-1vmg"
server_key: "50e46bf0c5ee8ae4c174ad75790e8b5b"
server_key_iv: "b6a2b8723d74707e90adb19e77e2e570"
server_key_version: "encryptionkey@0"
I20260811 15:08:20.734287 32747 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 15:08:20.737511 301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:20.739239 32747 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:08:20.739387 32747 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "7f1c581731e14b9fa50634b9a50957e1"
format_stamp: "Formatted at 2026-08-11 15:08:20 on dist-test-slave-1vmg"
server_key: "50e46bf0c5ee8ae4c174ad75790e8b5b"
server_key_iv: "b6a2b8723d74707e90adb19e77e2e570"
server_key_version: "encryptionkey@0"
I20260811 15:08:20.739521 32747 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:20.759796 32747 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:20.763031 32747 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:20.763235 32747 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:20.764791 32747 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:20.764848 32747 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:20.764894 32747 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:20.764922 32747 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 15:08:20 dist-test-slave-1vmg krb5kdc[32542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460900, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:20.775446 32747 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:36389
I20260811 15:08:20.775533 503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:36389 every 8 connection(s)
I20260811 15:08:20.776589 32747 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:08:20.778860 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:20.765646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:44069 (local address 127.30.8.62:43365)
0811 15:08:20.765812 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:20.765817 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:20.766686 (+ 869us) server_negotiation.cc:407] Connection header received
0811 15:08:20.767831 (+ 1145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:20.767836 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:20.767899 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:20.767960 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:20.769759 (+ 1799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:20.770573 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:20.771901 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:20.772159 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:20.772259 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:20.775248 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:20.775253 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:20.775256 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:20.775291 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:20.776750 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:20.777362 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:20.777365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:20.777367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:20.777463 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:20.777837 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:20.777839 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:20.777840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:20.778070 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:20.778149 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:20.778614 (+ 465us) server_negotiation.cc:299] Negotiation successful
0811 15:08:20.778734 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 15:08:20.779104 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 32747
I20260811 15:08:20.779232 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:08:20.779521 30752 external_mini_cluster.cc:1428] Setting key 7ace41daefc4a0ceeb5e875f5324a171
I20260811 15:08:20.780021 315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:20.766016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:43365 (local address 127.30.8.1:44069)
0811 15:08:20.766499 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:20.766552 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:20.767556 (+ 1004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:20.768424 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:20.768434 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:20.768870 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:20.769596 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:20.769618 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:20.770699 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:20.770702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:20.771774 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:20.771785 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:20.772320 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:20.772324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:20.772508 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:20.773205 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:20.773230 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:20.775017 (+ 1787us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:20.776856 (+ 1839us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:20.776863 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:20.776879 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:20.777238 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:20.777562 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:20.777565 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:20.777567 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:20.777721 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:20.778149 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:20.778154 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:20.778408 (+ 254us) client_negotiation.cc:769] Sending connection context
0811 15:08:20.778569 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 15:08:20.778933 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":125,"threads_started":1}
I20260811 15:08:20.781574 504 heartbeater.cc:347] Connected to a master server at 127.30.8.62:43365
I20260811 15:08:20.781730 504 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:20.782341 504 heartbeater.cc:511] Master 127.30.8.62:43365 requested a full tablet report, sending...
I20260811 15:08:20.785578 32612 ts_manager.cc:194] Registered new tserver with Master: 7f1c581731e14b9fa50634b9a50957e1 (127.30.8.1:36389)
I20260811 15:08:20.786803 32612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:44069
WARNING: no policy specified for kudu/127.30.8.2@KRBTEST.COM; defaulting to no policy
I20260811 15:08:20.794193 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:20.784233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:38043 (local address 127.30.8.62:43365)
0811 15:08:20.784362 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:20.784367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:20.784946 (+ 579us) server_negotiation.cc:407] Connection header received
0811 15:08:20.785092 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:20.785095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:20.785173 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:20.785284 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:20.785804 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:20.786896 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:20.788422 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:20.788772 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:20.788844 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:20.790369 (+ 1525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:20.790373 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:20.790375 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:20.790401 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:20.792136 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:20.792685 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:20.792690 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:20.792692 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:20.792785 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:20.793113 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:20.793116 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:20.793118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:20.793362 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:20.793437 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:20.793838 (+ 401us) server_negotiation.cc:299] Negotiation successful
0811 15:08:20.793988 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":53}
I20260811 15:08:20.794750 512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:20.784470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:43365 (local address 127.30.8.1:38043)
0811 15:08:20.784864 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:20.784878 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:20.784983 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:20.785302 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:20.785305 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:20.785538 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:20.785679 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:20.785688 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:20.787134 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:20.787138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:20.788264 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:20.788276 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:20.789048 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:20.789051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:20.789141 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:20.789728 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:20.789743 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:20.790211 (+ 468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:20.792290 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:20.792294 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:20.792296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:20.792561 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:20.792901 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:20.792904 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:20.792906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:20.792997 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:20.793514 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:20.793517 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:20.793633 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 15:08:20.794371 (+ 738us) client_negotiation.cc:240] Negotiation successful
0811 15:08:20.794550 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":120,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
I20260811 15:08:20.818888 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:43365
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:41899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:20.951256 515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:20.951625 515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:20.951676 515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:20.956032 515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:20.956123 515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:20.956174 515 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 15:08:20.956238 515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:08:20.962376 515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:41899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.8.62:43365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:20.963886 515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:20.965204 515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:20.977058 522 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:20.977079 525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:20.977260 515 server_base.cc:1034] running on GCE node
W20260811 15:08:20.977257 529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:20.977993 515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:20.978765 515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:20.980023 515 hybrid_clock.cc:648] HybridClock initialized: now 1786460900980000 us; error 54 us; skew 500 ppm
Aug 11 15:08:20 dist-test-slave-1vmg krb5kdc[32542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460900, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:20.983985 515 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:08:20.985394 515 webserver.cc:460] Webserver started at http://127.30.8.2:42061/ using document root <none> and password file <none>
I20260811 15:08:20.986116 515 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:20.986189 515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:20.986439 515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:20.989164 515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "f660d809fcb94aec94e3d9641c5193ec"
format_stamp: "Formatted at 2026-08-11 15:08:20 on dist-test-slave-1vmg"
server_key: "946a69d508a09c105eb8ebec85262a68"
server_key_iv: "c036f198cbc4e8617629f3641952e75e"
server_key_version: "encryptionkey@0"
I20260811 15:08:20.989811 515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "f660d809fcb94aec94e3d9641c5193ec"
format_stamp: "Formatted at 2026-08-11 15:08:20 on dist-test-slave-1vmg"
server_key: "946a69d508a09c105eb8ebec85262a68"
server_key_iv: "c036f198cbc4e8617629f3641952e75e"
server_key_version: "encryptionkey@0"
I20260811 15:08:20.994782 515 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 15:08:20.997867 542 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:20.999373 515 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:08:20.999524 515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "f660d809fcb94aec94e3d9641c5193ec"
format_stamp: "Formatted at 2026-08-11 15:08:20 on dist-test-slave-1vmg"
server_key: "946a69d508a09c105eb8ebec85262a68"
server_key_iv: "c036f198cbc4e8617629f3641952e75e"
server_key_version: "encryptionkey@0"
I20260811 15:08:20.999665 515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:21.020884 515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:21.024619 515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:21.024818 515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:21.026563 515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:21.026625 515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:21.026682 515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:21.026707 515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 15:08:21 dist-test-slave-1vmg krb5kdc[32542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460900, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:21.038189 515 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:38015
I20260811 15:08:21.038223 741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:38015 every 8 connection(s)
I20260811 15:08:21.039343 515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:08:21.041474 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.027332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:54303 (local address 127.30.8.62:43365)
0811 15:08:21.027473 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:21.027478 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:21.029383 (+ 1905us) server_negotiation.cc:407] Connection header received
0811 15:08:21.030618 (+ 1235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:21.030622 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:21.030683 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:21.030772 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:21.032588 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.033307 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.034583 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.034831 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.034947 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.037673 (+ 2726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:21.037677 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:21.037679 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:21.037707 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:21.039321 (+ 1614us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:21.039973 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:21.039977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:21.039979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:21.040081 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:21.040447 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:21.040464 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:21.040466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:21.040688 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:21.040772 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:21.041236 (+ 464us) server_negotiation.cc:299] Negotiation successful
0811 15:08:21.041348 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 15:08:21.042361 558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.028702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:43365 (local address 127.30.8.2:54303)
0811 15:08:21.029208 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:21.029265 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:21.030376 (+ 1111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:21.031025 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:21.031036 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:21.031658 (+ 622us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:21.032429 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:21.032451 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.033439 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.033443 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:21.034457 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:21.034469 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.035010 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.035015 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:21.035207 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.035809 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:21.035827 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:21.037468 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:21.039439 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:21.039446 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:21.039462 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:21.039850 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:21.040186 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:21.040189 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:21.040191 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:21.040358 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:21.040779 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:21.040785 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:21.041034 (+ 249us) client_negotiation.cc:769] Sending connection context
0811 15:08:21.041204 (+ 170us) client_negotiation.cc:240] Negotiation successful
0811 15:08:21.041472 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":140,"threads_started":1}
I20260811 15:08:21.043594 742 heartbeater.cc:347] Connected to a master server at 127.30.8.62:43365
I20260811 15:08:21.043751 742 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:21.044387 742 heartbeater.cc:511] Master 127.30.8.62:43365 requested a full tablet report, sending...
I20260811 15:08:21.045583 32615 ts_manager.cc:194] Registered new tserver with Master: f660d809fcb94aec94e3d9641c5193ec (127.30.8.2:38015)
I20260811 15:08:21.046351 32615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:54303
I20260811 15:08:21.049116 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 515
I20260811 15:08:21.049233 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:08:21.049503 30752 external_mini_cluster.cc:1428] Setting key be4043ff228ab63a7492c1c6af0c0042
I20260811 15:08:21.053619 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.046131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:44817 (local address 127.30.8.62:43365)
0811 15:08:21.046247 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:21.046252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:21.046415 (+ 163us) server_negotiation.cc:407] Connection header received
0811 15:08:21.046540 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:21.046543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:21.046590 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:21.046658 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:21.047358 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.048081 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.048833 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.049129 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.049238 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.050521 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:21.050524 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:21.050526 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:21.050549 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:21.052017 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:21.052410 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:21.052414 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:21.052415 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:21.052504 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:21.052720 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:21.052723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:21.052725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:21.053001 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:21.053115 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:21.053388 (+ 273us) server_negotiation.cc:299] Negotiation successful
0811 15:08:21.053486 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 15:08:21.053689 749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.046067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:43365 (local address 127.30.8.2:44817)
0811 15:08:21.046342 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:21.046354 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:21.046450 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:21.046908 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:21.046931 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:21.047140 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:21.047249 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:21.047255 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.048187 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.048189 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:21.048740 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:21.048747 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.049302 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.049306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:21.049385 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.049938 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:21.049953 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:21.050386 (+ 433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:21.052131 (+ 1745us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:21.052134 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:21.052135 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:21.052328 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:21.052581 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:21.052583 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:21.052585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:21.052653 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:21.053103 (+ 450us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:21.053106 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:21.053215 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 15:08:21.053326 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 15:08:21.053482 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for kudu/127.30.8.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
I20260811 15:08:21.080539 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:43365
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:41899
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:21.206027 753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:21.206358 753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:21.206425 753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:21.210697 753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:21.210794 753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:21.210853 753 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 15:08:21.210932 753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:08:21.216818 753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:41899
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.8.62:43365
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:21.218124 753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:21.219293 753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:21.229645 766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:21.229647 764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:21.229645 769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:21.230039 753 server_base.cc:1034] running on GCE node
I20260811 15:08:21.230491 753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:21.231290 753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:21.232458 753 hybrid_clock.cc:648] HybridClock initialized: now 1786460901232435 us; error 38 us; skew 500 ppm
Aug 11 15:08:21 dist-test-slave-1vmg krb5kdc[32542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460901, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:21.235520 753 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:08:21.236562 753 webserver.cc:460] Webserver started at http://127.30.8.3:38481/ using document root <none> and password file <none>
I20260811 15:08:21.237248 753 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:21.237316 753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:21.237504 753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:21.239814 753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "a170158c4d8e456ba756a6bbd95120e1"
format_stamp: "Formatted at 2026-08-11 15:08:21 on dist-test-slave-1vmg"
server_key: "80fae6b9670c6f0647dd8c30f807a789"
server_key_iv: "6af9df4642d1c0e1def10e9eb738bcbf"
server_key_version: "encryptionkey@0"
I20260811 15:08:21.240388 753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "a170158c4d8e456ba756a6bbd95120e1"
format_stamp: "Formatted at 2026-08-11 15:08:21 on dist-test-slave-1vmg"
server_key: "80fae6b9670c6f0647dd8c30f807a789"
server_key_iv: "6af9df4642d1c0e1def10e9eb738bcbf"
server_key_version: "encryptionkey@0"
I20260811 15:08:21.244957 753 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 15:08:21.247643 787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:21.248917 753 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 15:08:21.249050 753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "a170158c4d8e456ba756a6bbd95120e1"
format_stamp: "Formatted at 2026-08-11 15:08:21 on dist-test-slave-1vmg"
server_key: "80fae6b9670c6f0647dd8c30f807a789"
server_key_iv: "6af9df4642d1c0e1def10e9eb738bcbf"
server_key_version: "encryptionkey@0"
I20260811 15:08:21.249191 753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:21.262879 753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:21.265978 753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:21.266153 753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:21.267758 753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:21.267820 753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:21.267865 753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:21.267891 753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:21.278831 753 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:35451
I20260811 15:08:21.278900 984 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:35451 every 8 connection(s)
Aug 11 15:08:21 dist-test-slave-1vmg krb5kdc[32542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460901, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:21.280025 753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:08:21.283779 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.268606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:51403 (local address 127.30.8.62:43365)
0811 15:08:21.268770 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:21.268775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:21.270352 (+ 1577us) server_negotiation.cc:407] Connection header received
0811 15:08:21.271860 (+ 1508us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:21.271865 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:21.271942 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:21.272086 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:21.273960 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.274780 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.276374 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.276703 (+ 329us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.276825 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.279924 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:21.279928 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:21.279930 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:21.279958 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:21.281488 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:21.282065 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:21.282069 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:21.282071 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:21.282168 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:21.282534 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:21.282538 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:21.282540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:21.282741 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:21.282829 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:21.283435 (+ 606us) server_negotiation.cc:299] Negotiation successful
0811 15:08:21.283580 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 15:08:21.284660 806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.269297 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:43365 (local address 127.30.8.3:51403)
0811 15:08:21.270084 (+ 787us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:21.270153 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:21.271538 (+ 1385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:21.272365 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:21.272376 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:21.272944 (+ 568us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:21.273735 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:21.273763 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.274942 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.274947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:21.276228 (+ 1281us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:21.276240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.276905 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.276910 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:21.277147 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.277909 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:21.277928 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:21.279714 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:21.281618 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:21.281624 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:21.281642 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:21.281949 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:21.282255 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:21.282258 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:21.282260 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:21.282423 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:21.282844 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:21.282852 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:21.283181 (+ 329us) client_negotiation.cc:769] Sending connection context
0811 15:08:21.283358 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 15:08:21.283732 (+ 374us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":211,"threads_started":1}
I20260811 15:08:21.285878 985 heartbeater.cc:347] Connected to a master server at 127.30.8.62:43365
I20260811 15:08:21.286038 985 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:21.286584 985 heartbeater.cc:511] Master 127.30.8.62:43365 requested a full tablet report, sending...
I20260811 15:08:21.287707 32612 ts_manager.cc:194] Registered new tserver with Master: a170158c4d8e456ba756a6bbd95120e1 (127.30.8.3:35451)
I20260811 15:08:21.288359 32612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:51403
I20260811 15:08:21.290017 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 753
I20260811 15:08:21.290139 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:08:21.290429 30752 external_mini_cluster.cc:1428] Setting key aad0cc934d26452c6df7a61ad22d8da3
I20260811 15:08:21.292562 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 15:08:21.295941 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.288213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:38723 (local address 127.30.8.62:43365)
0811 15:08:21.288335 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:21.288340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:21.288495 (+ 155us) server_negotiation.cc:407] Connection header received
0811 15:08:21.288633 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:21.288636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:21.288686 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:21.288765 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:21.289256 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.290040 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.291037 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.291342 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.291428 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.292698 (+ 1270us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:21.292702 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:21.292705 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:21.292731 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:21.294222 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:21.294686 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:21.294689 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:21.294691 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:21.294786 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:21.295104 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:21.295108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:21.295110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:21.295301 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:21.295394 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:21.295663 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 15:08:21.295778 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 15:08:21.295948 995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.288121 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:43365 (local address 127.30.8.3:38723)
0811 15:08:21.288420 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:21.288436 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:21.288541 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:21.288797 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:21.288800 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:21.289029 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:21.289143 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:21.289150 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.290201 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.290205 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:21.290904 (+ 699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:21.290940 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.291457 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.291461 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:21.291541 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.292103 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:21.292116 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:21.292561 (+ 445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:21.294345 (+ 1784us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:21.294348 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:21.294350 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:21.294596 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:21.294869 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:21.294872 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:21.294874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:21.295004 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:21.295396 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:21.295399 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:21.295496 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 15:08:21.295593 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 15:08:21.295762 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":104,"threads_started":1}
Aug 11 15:08:21 dist-test-slave-1vmg krb5kdc[32542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460901, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 15:08:21 dist-test-slave-1vmg krb5kdc[32542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460901, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:21.322391 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.313157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49654 (local address 127.30.8.62:43365)
0811 15:08:21.313292 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:21.313296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:21.313310 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:08:21.313356 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:21.313358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:21.313399 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:21.313469 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:21.314209 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.314743 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.315608 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.315829 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.315924 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.319293 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:21.319297 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:21.319299 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:21.319335 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:21.320783 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:21.321246 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:21.321249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:21.321250 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:21.321330 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:21.321547 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:21.321549 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:21.321550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:21.321754 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:21.321860 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:21.322139 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 15:08:21.322256 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 15:08:21.326679 32612 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:08:21.331085 32612 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:08:21.346160 1016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.340098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:35451 (local address 127.0.0.1:52680)
0811 15:08:21.340621 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:21.340663 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:21.340802 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:21.342116 (+ 1314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:21.342119 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:21.342138 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:21.342233 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:21.342239 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.343357 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.343360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:21.344719 (+ 1359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:21.344731 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.345569 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.345573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:21.345692 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.345737 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 15:08:21.345852 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 15:08:21.345985 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":62,"thread_start_us":130,"threads_started":1}
I20260811 15:08:21.346199 1015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.340127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45856 (local address 127.30.8.1:36389)
0811 15:08:21.340439 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:21.340447 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:21.340744 (+ 297us) server_negotiation.cc:407] Connection header received
0811 15:08:21.341007 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:21.341014 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:21.341235 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:21.341361 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:21.341882 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.343042 (+ 1160us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.344522 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.345330 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.345411 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.345582 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:21.345941 (+ 359us) server_negotiation.cc:299] Negotiation successful
0811 15:08:21.346055 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":101,"threads_started":1}
I20260811 15:08:21.346256 1022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.341215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46458 (local address 127.30.8.2:38015)
0811 15:08:21.341662 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:21.341673 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:21.341691 (+ 18us) server_negotiation.cc:407] Connection header received
0811 15:08:21.341761 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:21.341766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:21.341944 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:21.342093 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:21.342493 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.343599 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.344981 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.345741 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.345843 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.345995 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:21.346042 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 15:08:21.346130 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":114,"threads_started":1}
I20260811 15:08:21.346170 1020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.340766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:38015 (local address 127.0.0.1:46458)
0811 15:08:21.341250 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:21.341262 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:21.341344 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:21.342259 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:21.342262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:21.342279 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:21.342366 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:21.342371 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.343728 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.343731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:21.344884 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:21.344892 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.345855 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.345859 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:21.345939 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.345959 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 15:08:21.346013 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 15:08:21.346067 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":108,"threads_started":1}
I20260811 15:08:21.346606 1013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.339981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:36389 (local address 127.0.0.1:45856)
0811 15:08:21.340610 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:21.340662 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:21.340821 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:21.341558 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:21.341576 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:21.341600 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:21.341746 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:21.341754 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.343194 (+ 1440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.343197 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:21.344397 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:21.344406 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.345686 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.345689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:21.345781 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.345807 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 15:08:21.346405 (+ 598us) client_negotiation.cc:240] Negotiation successful
0811 15:08:21.346472 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":3,"thread_start_us":108,"threads_started":1}
I20260811 15:08:21.347059 1021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.340845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52680 (local address 127.30.8.3:35451)
0811 15:08:21.341647 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:21.341663 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:21.341687 (+ 24us) server_negotiation.cc:407] Connection header received
0811 15:08:21.341761 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:21.341766 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:21.341944 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:21.342119 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:21.342349 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.343248 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.344861 (+ 1613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.345450 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.345552 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.345713 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:21.346781 (+ 1068us) server_negotiation.cc:299] Negotiation successful
0811 15:08:21.346888 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":569,"thread_start_us":106,"threads_started":1}
I20260811 15:08:21.349673 392 tablet_service.cc:1467] Processing CreateTablet for tablet 2da47be1c6324d2aa22495778797b609 (DEFAULT_TABLE table=test-table [id=ca1cdce3962c4e57bccd8742a00397fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:21.349699 612 tablet_service.cc:1467] Processing CreateTablet for tablet 2da47be1c6324d2aa22495778797b609 (DEFAULT_TABLE table=test-table [id=ca1cdce3962c4e57bccd8742a00397fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:21.349700 869 tablet_service.cc:1467] Processing CreateTablet for tablet 2da47be1c6324d2aa22495778797b609 (DEFAULT_TABLE table=test-table [id=ca1cdce3962c4e57bccd8742a00397fb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:21.350751 392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2da47be1c6324d2aa22495778797b609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:21.350769 612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2da47be1c6324d2aa22495778797b609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:21.350782 869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2da47be1c6324d2aa22495778797b609. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:21.357663 1023 tablet_bootstrap.cc:492] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1: Bootstrap starting.
I20260811 15:08:21.359094 1024 tablet_bootstrap.cc:492] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1: Bootstrap starting.
I20260811 15:08:21.360633 1023 tablet_bootstrap.cc:654] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:21.361152 1024 tablet_bootstrap.cc:654] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:21.361594 1023 log.cc:824] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:21.361927 1024 log.cc:824] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:21.362421 1025 tablet_bootstrap.cc:492] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec: Bootstrap starting.
I20260811 15:08:21.363864 1023 tablet_bootstrap.cc:492] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1: No bootstrap required, opened a new log
I20260811 15:08:21.363968 1024 tablet_bootstrap.cc:492] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1: No bootstrap required, opened a new log
I20260811 15:08:21.364078 1023 ts_tablet_manager.cc:1397] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260811 15:08:21.364164 1024 ts_tablet_manager.cc:1397] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 15:08:21.364658 1025 tablet_bootstrap.cc:654] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:21.365491 1025 log.cc:824] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:21.367785 1025 tablet_bootstrap.cc:492] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec: No bootstrap required, opened a new log
I20260811 15:08:21.367684 1023 raft_consensus.cc:348] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } }
I20260811 15:08:21.367930 1023 raft_consensus.cc:374] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:21.367982 1023 raft_consensus.cc:729] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f1c581731e14b9fa50634b9a50957e1, State: Initialized, Role: FOLLOWER
I20260811 15:08:21.367996 1025 ts_tablet_manager.cc:1397] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 15:08:21.367986 1024 raft_consensus.cc:348] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } }
I20260811 15:08:21.368233 1024 raft_consensus.cc:374] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:21.368300 1024 raft_consensus.cc:729] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a170158c4d8e456ba756a6bbd95120e1, State: Initialized, Role: FOLLOWER
I20260811 15:08:21.368453 1023 consensus_queue.cc:260] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } }
I20260811 15:08:21.368705 1024 consensus_queue.cc:260] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } }
I20260811 15:08:21.369215 504 heartbeater.cc:503] Master 127.30.8.62:43365 was elected leader, sending a full tablet report...
I20260811 15:08:21.369491 1023 ts_tablet_manager.cc:1428] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 15:08:21.369635 1024 ts_tablet_manager.cc:1428] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 15:08:21.369973 985 heartbeater.cc:503] Master 127.30.8.62:43365 was elected leader, sending a full tablet report...
I20260811 15:08:21.373142 1025 raft_consensus.cc:348] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } }
I20260811 15:08:21.373445 1025 raft_consensus.cc:374] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:21.373517 1025 raft_consensus.cc:729] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f660d809fcb94aec94e3d9641c5193ec, State: Initialized, Role: FOLLOWER
I20260811 15:08:21.373988 1025 consensus_queue.cc:260] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } }
I20260811 15:08:21.374738 742 heartbeater.cc:503] Master 127.30.8.62:43365 was elected leader, sending a full tablet report...
I20260811 15:08:21.375162 1025 ts_tablet_manager.cc:1428] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 15:08:21.527199 1034 raft_consensus.cc:482] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:08:21.527544 1034 raft_consensus.cc:504] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } }
W20260811 15:08:21.528000 505 tablet.cc:2369] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:08:21.529130 1034 leader_election.cc:288] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f1c581731e14b9fa50634b9a50957e1 (127.30.8.1:36389), f660d809fcb94aec94e3d9641c5193ec (127.30.8.2:38015)
W20260811 15:08:21.531793 989 tablet.cc:2369] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:08:21.534045 1015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.529625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:48219 (local address 127.30.8.1:36389)
0811 15:08:21.529753 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:21.529757 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:21.529812 (+ 55us) server_negotiation.cc:407] Connection header received
0811 15:08:21.530023 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:21.530026 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:21.530080 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:21.530148 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:21.530581 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.531317 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.532748 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.533403 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.533527 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.533598 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:21.533831 (+ 233us) server_negotiation.cc:299] Negotiation successful
0811 15:08:21.533904 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 15:08:21.534045 1022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.530043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:41497 (local address 127.30.8.2:38015)
0811 15:08:21.530186 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:21.530190 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:21.530316 (+ 126us) server_negotiation.cc:407] Connection header received
0811 15:08:21.530438 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:21.530441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:21.530488 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:21.530557 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:21.530838 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.531477 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.532809 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.533428 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.533526 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.533602 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:21.533827 (+ 225us) server_negotiation.cc:299] Negotiation successful
0811 15:08:21.533904 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 15:08:21.534113 806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.529518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:36389 (local address 127.30.8.3:48219)
0811 15:08:21.529729 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:21.529748 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:21.529883 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:21.530275 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:21.530278 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:21.530298 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:21.530457 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:21.530464 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.531414 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.531418 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:21.532625 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:21.532635 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.533531 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.533535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:21.533719 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.533735 (+ 16us) client_negotiation.cc:769] Sending connection context
0811 15:08:21.533813 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 15:08:21.533924 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"mutex_wait_us":127,"spinlock_wait_cycles":44928}
I20260811 15:08:21.534473 1036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.529836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:38015 (local address 127.30.8.3:41497)
0811 15:08:21.530226 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:21.530238 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:21.530349 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:21.530585 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:21.530588 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:21.530604 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:21.530730 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:21.530736 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.531614 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.531617 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:21.532702 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:21.532713 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.533531 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.533535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:21.533698 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.533735 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 15:08:21.533813 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 15:08:21.533924 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"mutex_wait_us":54,"thread_start_us":145,"threads_started":1}
I20260811 15:08:21.534819 424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2da47be1c6324d2aa22495778797b609" candidate_uuid: "a170158c4d8e456ba756a6bbd95120e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f1c581731e14b9fa50634b9a50957e1" is_pre_election: true
I20260811 15:08:21.535091 658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2da47be1c6324d2aa22495778797b609" candidate_uuid: "a170158c4d8e456ba756a6bbd95120e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f660d809fcb94aec94e3d9641c5193ec" is_pre_election: true
I20260811 15:08:21.535202 424 raft_consensus.cc:2454] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a170158c4d8e456ba756a6bbd95120e1 in term 0.
I20260811 15:08:21.535365 658 raft_consensus.cc:2454] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a170158c4d8e456ba756a6bbd95120e1 in term 0.
I20260811 15:08:21.535704 795 leader_election.cc:302] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f1c581731e14b9fa50634b9a50957e1, a170158c4d8e456ba756a6bbd95120e1; no voters:
I20260811 15:08:21.535995 1034 raft_consensus.cc:2781] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:08:21.536087 1034 raft_consensus.cc:482] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:08:21.536140 1034 raft_consensus.cc:3037] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:21.537361 1034 raft_consensus.cc:504] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } }
I20260811 15:08:21.537725 1034 leader_election.cc:288] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [CANDIDATE]: Term 1 election: Requested vote from peers 7f1c581731e14b9fa50634b9a50957e1 (127.30.8.1:36389), f660d809fcb94aec94e3d9641c5193ec (127.30.8.2:38015)
I20260811 15:08:21.538178 658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2da47be1c6324d2aa22495778797b609" candidate_uuid: "a170158c4d8e456ba756a6bbd95120e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f660d809fcb94aec94e3d9641c5193ec"
I20260811 15:08:21.538178 424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2da47be1c6324d2aa22495778797b609" candidate_uuid: "a170158c4d8e456ba756a6bbd95120e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f1c581731e14b9fa50634b9a50957e1"
I20260811 15:08:21.538321 424 raft_consensus.cc:3037] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:21.538321 658 raft_consensus.cc:3037] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:21.539697 658 raft_consensus.cc:2454] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a170158c4d8e456ba756a6bbd95120e1 in term 1.
I20260811 15:08:21.539705 424 raft_consensus.cc:2454] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a170158c4d8e456ba756a6bbd95120e1 in term 1.
I20260811 15:08:21.540184 795 leader_election.cc:302] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f1c581731e14b9fa50634b9a50957e1, a170158c4d8e456ba756a6bbd95120e1; no voters:
I20260811 15:08:21.540596 1034 raft_consensus.cc:2781] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [term 1 FOLLOWER]: Leader election won for term 1
W20260811 15:08:21.540767 744 tablet.cc:2369] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:08:21.540858 1034 raft_consensus.cc:686] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [term 1 LEADER]: Becoming Leader. State: Replica: a170158c4d8e456ba756a6bbd95120e1, State: Running, Role: LEADER
I20260811 15:08:21.541397 1034 consensus_queue.cc:237] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } }
I20260811 15:08:21.544935 32612 catalog_manager.cc:5495] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a170158c4d8e456ba756a6bbd95120e1 (127.30.8.3). New cstate: current_term: 1 leader_uuid: "a170158c4d8e456ba756a6bbd95120e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } health_report { overall_health: UNKNOWN } } }
I20260811 15:08:21.593465 1021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.589903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52684 (local address 127.30.8.3:35451)
0811 15:08:21.590027 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:21.590031 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:21.590046 (+ 15us) server_negotiation.cc:407] Connection header received
0811 15:08:21.590237 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:21.590240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:21.590304 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:21.590384 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:21.590611 (+ 227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.591188 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.592018 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.592235 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.592301 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.592586 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:21.593059 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:21.593165 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:21.593224 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 15:08:21.593284 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 15:08:21.599618 658 raft_consensus.cc:1261] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec [term 1 FOLLOWER]: Refusing update from remote peer a170158c4d8e456ba756a6bbd95120e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:21.599634 424 raft_consensus.cc:1261] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1 [term 1 FOLLOWER]: Refusing update from remote peer a170158c4d8e456ba756a6bbd95120e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:21.600335 1034 consensus_queue.cc:1035] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:21.600551 1039 consensus_queue.cc:1035] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:21.611253 1043 mvcc.cc:205] Tried to move back new op lower bound from 7317343852944486400 to 7317343852716597248. Current Snapshot: MvccSnapshot[applied={T|T < 7317343852944486400}]
I20260811 15:08:21.611457 1042 mvcc.cc:205] Tried to move back new op lower bound from 7317343852944486400 to 7317343852716597248. Current Snapshot: MvccSnapshot[applied={T|T < 7317343852944486400 or (T in {7317343852944486400})}]
I20260811 15:08:21.623291 32612 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:21.623620 32612 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:21.627149 32612 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c: Sending DeleteTablet for 3 replicas of tablet 2da47be1c6324d2aa22495778797b609
I20260811 15:08:21.628456 869 tablet_service.cc:1514] Processing DeleteTablet for tablet 2da47be1c6324d2aa22495778797b609 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:21 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:52680
I20260811 15:08:21.628533 612 tablet_service.cc:1514] Processing DeleteTablet for tablet 2da47be1c6324d2aa22495778797b609 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:21 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:46458
I20260811 15:08:21.628700 392 tablet_service.cc:1514] Processing DeleteTablet for tablet 2da47be1c6324d2aa22495778797b609 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:21 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:45856
I20260811 15:08:21.629098 1058 tablet_replica.cc:330] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec: stopping tablet replica
I20260811 15:08:21.629383 1059 tablet_replica.cc:330] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1: stopping tablet replica
I20260811 15:08:21.629489 1058 raft_consensus.cc:2229] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:21.629804 1059 raft_consensus.cc:2229] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:08:21.630105 1060 tablet_replica.cc:330] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1: stopping tablet replica
I20260811 15:08:21.630138 32612 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49654:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 15:08:21.630398 1060 raft_consensus.cc:2229] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:21.630595 1059 raft_consensus.cc:2258] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:21.630666 1060 raft_consensus.cc:2258] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 15:08:21.630657 32612 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:08:21.629860 1058 raft_consensus.cc:2258] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:21.632580 1058 ts_tablet_manager.cc:1905] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:21.632575 1060 ts_tablet_manager.cc:1905] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:21.635434 1058 ts_tablet_manager.cc:1918] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:08:21.635434 1060 ts_tablet_manager.cc:1918] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:08:21.635593 1060 log.cc:1196] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-0/wal/wals/2da47be1c6324d2aa22495778797b609
I20260811 15:08:21.635622 1058 log.cc:1196] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/2da47be1c6324d2aa22495778797b609
I20260811 15:08:21.635986 1058 ts_tablet_manager.cc:1939] T 2da47be1c6324d2aa22495778797b609 P f660d809fcb94aec94e3d9641c5193ec: Deleting consensus metadata
I20260811 15:08:21.635985 1060 ts_tablet_manager.cc:1939] T 2da47be1c6324d2aa22495778797b609 P 7f1c581731e14b9fa50634b9a50957e1: Deleting consensus metadata
I20260811 15:08:21.637418 32593 catalog_manager.cc:4841] TS f660d809fcb94aec94e3d9641c5193ec (127.30.8.2:38015): tablet 2da47be1c6324d2aa22495778797b609 (table test-table [id=ca1cdce3962c4e57bccd8742a00397fb]) successfully deleted
I20260811 15:08:21.637629 32593 catalog_manager.cc:4841] TS 7f1c581731e14b9fa50634b9a50957e1 (127.30.8.1:36389): tablet 2da47be1c6324d2aa22495778797b609 (table test-table [id=ca1cdce3962c4e57bccd8742a00397fb]) successfully deleted
I20260811 15:08:21.638391 612 tablet_service.cc:1467] Processing CreateTablet for tablet 906e7ec82992494b8a3c6d6cc29511b4 (DEFAULT_TABLE table=test-table [id=9514d744e7ee4d81ab0062d2cee20883]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:21.638391 392 tablet_service.cc:1467] Processing CreateTablet for tablet 906e7ec82992494b8a3c6d6cc29511b4 (DEFAULT_TABLE table=test-table [id=9514d744e7ee4d81ab0062d2cee20883]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:21.638671 392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 906e7ec82992494b8a3c6d6cc29511b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:21.638671 612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 906e7ec82992494b8a3c6d6cc29511b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:21.638890 869 tablet_service.cc:1467] Processing CreateTablet for tablet 906e7ec82992494b8a3c6d6cc29511b4 (DEFAULT_TABLE table=test-table [id=9514d744e7ee4d81ab0062d2cee20883]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:21.639230 869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 906e7ec82992494b8a3c6d6cc29511b4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:21.641067 1025 tablet_bootstrap.cc:492] T 906e7ec82992494b8a3c6d6cc29511b4 P f660d809fcb94aec94e3d9641c5193ec: Bootstrap starting.
I20260811 15:08:21.641767 1023 tablet_bootstrap.cc:492] T 906e7ec82992494b8a3c6d6cc29511b4 P 7f1c581731e14b9fa50634b9a50957e1: Bootstrap starting.
I20260811 15:08:21.641878 1024 tablet_bootstrap.cc:492] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1: Bootstrap starting.
I20260811 15:08:21.642230 1025 tablet_bootstrap.cc:654] T 906e7ec82992494b8a3c6d6cc29511b4 P f660d809fcb94aec94e3d9641c5193ec: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:21.642714 1023 tablet_bootstrap.cc:654] T 906e7ec82992494b8a3c6d6cc29511b4 P 7f1c581731e14b9fa50634b9a50957e1: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:21.642840 1024 tablet_bootstrap.cc:654] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:21.644213 1024 tablet_bootstrap.cc:492] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1: No bootstrap required, opened a new log
I20260811 15:08:21.644299 1025 tablet_bootstrap.cc:492] T 906e7ec82992494b8a3c6d6cc29511b4 P f660d809fcb94aec94e3d9641c5193ec: No bootstrap required, opened a new log
I20260811 15:08:21.644312 1024 ts_tablet_manager.cc:1397] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 15:08:21.644388 1025 ts_tablet_manager.cc:1397] T 906e7ec82992494b8a3c6d6cc29511b4 P f660d809fcb94aec94e3d9641c5193ec: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 15:08:21.644784 1024 raft_consensus.cc:348] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } }
I20260811 15:08:21.644912 1024 raft_consensus.cc:374] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:21.644941 1059 ts_tablet_manager.cc:1905] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:21.644958 1024 raft_consensus.cc:729] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a170158c4d8e456ba756a6bbd95120e1, State: Initialized, Role: FOLLOWER
I20260811 15:08:21.645246 1024 consensus_queue.cc:260] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } }
I20260811 15:08:21.645550 1024 ts_tablet_manager.cc:1428] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 15:08:21.644840 1025 raft_consensus.cc:348] T 906e7ec82992494b8a3c6d6cc29511b4 P f660d809fcb94aec94e3d9641c5193ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } }
I20260811 15:08:21.646042 1025 raft_consensus.cc:374] T 906e7ec82992494b8a3c6d6cc29511b4 P f660d809fcb94aec94e3d9641c5193ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:21.646098 1025 raft_consensus.cc:729] T 906e7ec82992494b8a3c6d6cc29511b4 P f660d809fcb94aec94e3d9641c5193ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f660d809fcb94aec94e3d9641c5193ec, State: Initialized, Role: FOLLOWER
I20260811 15:08:21.646272 1025 consensus_queue.cc:260] T 906e7ec82992494b8a3c6d6cc29511b4 P f660d809fcb94aec94e3d9641c5193ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } }
I20260811 15:08:21.647938 1059 ts_tablet_manager.cc:1918] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:08:21.648092 1059 log.cc:1196] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/2da47be1c6324d2aa22495778797b609
I20260811 15:08:21.648293 1025 ts_tablet_manager.cc:1428] T 906e7ec82992494b8a3c6d6cc29511b4 P f660d809fcb94aec94e3d9641c5193ec: Time spent starting tablet: real 0.004s user 0.001s sys 0.000s
I20260811 15:08:21.648362 1023 tablet_bootstrap.cc:492] T 906e7ec82992494b8a3c6d6cc29511b4 P 7f1c581731e14b9fa50634b9a50957e1: No bootstrap required, opened a new log
I20260811 15:08:21.648468 1023 ts_tablet_manager.cc:1397] T 906e7ec82992494b8a3c6d6cc29511b4 P 7f1c581731e14b9fa50634b9a50957e1: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 15:08:21.648655 1059 ts_tablet_manager.cc:1939] T 2da47be1c6324d2aa22495778797b609 P a170158c4d8e456ba756a6bbd95120e1: Deleting consensus metadata
I20260811 15:08:21.649120 1023 raft_consensus.cc:348] T 906e7ec82992494b8a3c6d6cc29511b4 P 7f1c581731e14b9fa50634b9a50957e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } }
I20260811 15:08:21.649256 1023 raft_consensus.cc:374] T 906e7ec82992494b8a3c6d6cc29511b4 P 7f1c581731e14b9fa50634b9a50957e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:21.649294 1023 raft_consensus.cc:729] T 906e7ec82992494b8a3c6d6cc29511b4 P 7f1c581731e14b9fa50634b9a50957e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f1c581731e14b9fa50634b9a50957e1, State: Initialized, Role: FOLLOWER
I20260811 15:08:21.649418 1023 consensus_queue.cc:260] T 906e7ec82992494b8a3c6d6cc29511b4 P 7f1c581731e14b9fa50634b9a50957e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } }
I20260811 15:08:21.649667 1023 ts_tablet_manager.cc:1428] T 906e7ec82992494b8a3c6d6cc29511b4 P 7f1c581731e14b9fa50634b9a50957e1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:21.650345 32594 catalog_manager.cc:4841] TS a170158c4d8e456ba756a6bbd95120e1 (127.30.8.3:35451): tablet 2da47be1c6324d2aa22495778797b609 (table test-table [id=ca1cdce3962c4e57bccd8742a00397fb]) successfully deleted
I20260811 15:08:21.706957 1034 raft_consensus.cc:482] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:08:21.707185 1034 raft_consensus.cc:504] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } }
I20260811 15:08:21.707757 1034 leader_election.cc:288] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f1c581731e14b9fa50634b9a50957e1 (127.30.8.1:36389), f660d809fcb94aec94e3d9641c5193ec (127.30.8.2:38015)
I20260811 15:08:21.708503 424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "906e7ec82992494b8a3c6d6cc29511b4" candidate_uuid: "a170158c4d8e456ba756a6bbd95120e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f1c581731e14b9fa50634b9a50957e1" is_pre_election: true
I20260811 15:08:21.708544 658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "906e7ec82992494b8a3c6d6cc29511b4" candidate_uuid: "a170158c4d8e456ba756a6bbd95120e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f660d809fcb94aec94e3d9641c5193ec" is_pre_election: true
I20260811 15:08:21.708693 424 raft_consensus.cc:2454] T 906e7ec82992494b8a3c6d6cc29511b4 P 7f1c581731e14b9fa50634b9a50957e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a170158c4d8e456ba756a6bbd95120e1 in term 0.
I20260811 15:08:21.708775 658 raft_consensus.cc:2454] T 906e7ec82992494b8a3c6d6cc29511b4 P f660d809fcb94aec94e3d9641c5193ec [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a170158c4d8e456ba756a6bbd95120e1 in term 0.
I20260811 15:08:21.709157 795 leader_election.cc:302] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f1c581731e14b9fa50634b9a50957e1, a170158c4d8e456ba756a6bbd95120e1; no voters:
I20260811 15:08:21.709522 1034 raft_consensus.cc:2781] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:08:21.709599 1034 raft_consensus.cc:482] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:08:21.709638 1034 raft_consensus.cc:3037] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:21.710690 1034 raft_consensus.cc:504] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } }
I20260811 15:08:21.711158 1034 leader_election.cc:288] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [CANDIDATE]: Term 1 election: Requested vote from peers 7f1c581731e14b9fa50634b9a50957e1 (127.30.8.1:36389), f660d809fcb94aec94e3d9641c5193ec (127.30.8.2:38015)
I20260811 15:08:21.711640 424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "906e7ec82992494b8a3c6d6cc29511b4" candidate_uuid: "a170158c4d8e456ba756a6bbd95120e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f1c581731e14b9fa50634b9a50957e1"
I20260811 15:08:21.711665 658 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "906e7ec82992494b8a3c6d6cc29511b4" candidate_uuid: "a170158c4d8e456ba756a6bbd95120e1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f660d809fcb94aec94e3d9641c5193ec"
I20260811 15:08:21.711802 658 raft_consensus.cc:3037] T 906e7ec82992494b8a3c6d6cc29511b4 P f660d809fcb94aec94e3d9641c5193ec [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:21.711805 424 raft_consensus.cc:3037] T 906e7ec82992494b8a3c6d6cc29511b4 P 7f1c581731e14b9fa50634b9a50957e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:21.713061 424 raft_consensus.cc:2454] T 906e7ec82992494b8a3c6d6cc29511b4 P 7f1c581731e14b9fa50634b9a50957e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a170158c4d8e456ba756a6bbd95120e1 in term 1.
I20260811 15:08:21.713132 658 raft_consensus.cc:2454] T 906e7ec82992494b8a3c6d6cc29511b4 P f660d809fcb94aec94e3d9641c5193ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a170158c4d8e456ba756a6bbd95120e1 in term 1.
I20260811 15:08:21.713622 795 leader_election.cc:302] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a170158c4d8e456ba756a6bbd95120e1, f660d809fcb94aec94e3d9641c5193ec; no voters:
I20260811 15:08:21.713999 1034 raft_consensus.cc:2781] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:21.714128 1034 raft_consensus.cc:686] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [term 1 LEADER]: Becoming Leader. State: Replica: a170158c4d8e456ba756a6bbd95120e1, State: Running, Role: LEADER
I20260811 15:08:21.714309 1034 consensus_queue.cc:237] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } }
I20260811 15:08:21.716401 32611 catalog_manager.cc:5495] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 reported cstate change: term changed from 0 to 1, leader changed from <none> to a170158c4d8e456ba756a6bbd95120e1 (127.30.8.3). New cstate: current_term: 1 leader_uuid: "a170158c4d8e456ba756a6bbd95120e1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } health_report { overall_health: HEALTHY } } }
Aug 11 15:08:21 dist-test-slave-1vmg krb5kdc[32542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460900, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:21.763178 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.752961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49658 (local address 127.30.8.62:43365)
0811 15:08:21.753117 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:21.753121 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:21.753408 (+ 287us) server_negotiation.cc:407] Connection header received
0811 15:08:21.753565 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:21.753567 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:21.753621 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:21.753688 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:21.754602 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.755177 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.756297 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.756525 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.756599 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.759419 (+ 2820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:21.759424 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:21.759427 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:21.759473 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:21.761141 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:21.761825 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:21.761830 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:21.761831 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:21.761938 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:21.762210 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:21.762212 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:21.762213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:21.762407 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:21.762483 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:21.762810 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 15:08:21.763050 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":109,"server-negotiator.queue_time_us":66}
I20260811 15:08:21.763197 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:21.752917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:43365 (local address 127.0.0.1:49658)
0811 15:08:21.753304 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:21.753321 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:21.753464 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:21.753747 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:21.753750 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:21.754359 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:21.754483 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:21.754491 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.755290 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.755294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:21.756175 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:21.756199 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:21.756636 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:21.756640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:21.756730 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:21.757373 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:21.757388 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:21.759093 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:21.761296 (+ 2203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:21.761308 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:21.761316 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:21.761681 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:21.762031 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:21.762034 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:21.762036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:21.762132 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:21.762495 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:21.762502 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:21.762647 (+ 145us) client_negotiation.cc:769] Sending connection context
0811 15:08:21.762761 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 15:08:21.762974 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":140,"threads_started":1}
I20260811 15:08:21.767956 32611 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:49658:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 15:08:21.768857 32611 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:08:21.774116 612 tablet_service.cc:1467] Processing CreateTablet for tablet fcae63eac50048c4aed0b8753edf76b8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ef043ca201254e688458f19c8af5ca51]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 15:08:21.774665 612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcae63eac50048c4aed0b8753edf76b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:21.776863 869 tablet_service.cc:1467] Processing CreateTablet for tablet fcae63eac50048c4aed0b8753edf76b8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ef043ca201254e688458f19c8af5ca51]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 15:08:21.777266 869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcae63eac50048c4aed0b8753edf76b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:21.778016 1025 tablet_bootstrap.cc:492] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec: Bootstrap starting.
I20260811 15:08:21.779131 1025 tablet_bootstrap.cc:654] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:21.774081 392 tablet_service.cc:1467] Processing CreateTablet for tablet fcae63eac50048c4aed0b8753edf76b8 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ef043ca201254e688458f19c8af5ca51]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 15:08:21.780227 392 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fcae63eac50048c4aed0b8753edf76b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:21.780817 1025 tablet_bootstrap.cc:492] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec: No bootstrap required, opened a new log
I20260811 15:08:21.781106 1025 ts_tablet_manager.cc:1397] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 15:08:21.781610 1025 raft_consensus.cc:348] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } }
I20260811 15:08:21.781735 1025 raft_consensus.cc:374] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:21.781772 1025 raft_consensus.cc:729] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f660d809fcb94aec94e3d9641c5193ec, State: Initialized, Role: FOLLOWER
I20260811 15:08:21.781899 1025 consensus_queue.cc:260] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } }
I20260811 15:08:21.782155 1025 ts_tablet_manager.cc:1428] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:21.784065 1035 tablet_replica.cc:439] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } } }
I20260811 15:08:21.784256 1035 tablet_replica.cc:442] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:21.785336 1024 tablet_bootstrap.cc:492] T fcae63eac50048c4aed0b8753edf76b8 P a170158c4d8e456ba756a6bbd95120e1: Bootstrap starting.
I20260811 15:08:21.786356 1024 tablet_bootstrap.cc:654] T fcae63eac50048c4aed0b8753edf76b8 P a170158c4d8e456ba756a6bbd95120e1: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:21.787642 1024 tablet_bootstrap.cc:492] T fcae63eac50048c4aed0b8753edf76b8 P a170158c4d8e456ba756a6bbd95120e1: No bootstrap required, opened a new log
I20260811 15:08:21.787730 1024 ts_tablet_manager.cc:1397] T fcae63eac50048c4aed0b8753edf76b8 P a170158c4d8e456ba756a6bbd95120e1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260811 15:08:21.787916 1023 tablet_bootstrap.cc:492] T fcae63eac50048c4aed0b8753edf76b8 P 7f1c581731e14b9fa50634b9a50957e1: Bootstrap starting.
I20260811 15:08:21.788115 1024 raft_consensus.cc:348] T fcae63eac50048c4aed0b8753edf76b8 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } }
I20260811 15:08:21.788218 1024 raft_consensus.cc:374] T fcae63eac50048c4aed0b8753edf76b8 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:21.788249 1024 raft_consensus.cc:729] T fcae63eac50048c4aed0b8753edf76b8 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a170158c4d8e456ba756a6bbd95120e1, State: Initialized, Role: FOLLOWER
I20260811 15:08:21.788372 1024 consensus_queue.cc:260] T fcae63eac50048c4aed0b8753edf76b8 P a170158c4d8e456ba756a6bbd95120e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } }
I20260811 15:08:21.788625 1024 ts_tablet_manager.cc:1428] T fcae63eac50048c4aed0b8753edf76b8 P a170158c4d8e456ba756a6bbd95120e1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:21.788798 1023 tablet_bootstrap.cc:654] T fcae63eac50048c4aed0b8753edf76b8 P 7f1c581731e14b9fa50634b9a50957e1: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:21.788750 1034 tablet_replica.cc:439] T fcae63eac50048c4aed0b8753edf76b8 P a170158c4d8e456ba756a6bbd95120e1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } } }
I20260811 15:08:21.788893 1034 tablet_replica.cc:442] T fcae63eac50048c4aed0b8753edf76b8 P a170158c4d8e456ba756a6bbd95120e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:21.790191 1023 tablet_bootstrap.cc:492] T fcae63eac50048c4aed0b8753edf76b8 P 7f1c581731e14b9fa50634b9a50957e1: No bootstrap required, opened a new log
I20260811 15:08:21.790277 1023 ts_tablet_manager.cc:1397] T fcae63eac50048c4aed0b8753edf76b8 P 7f1c581731e14b9fa50634b9a50957e1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 15:08:21.790665 1023 raft_consensus.cc:348] T fcae63eac50048c4aed0b8753edf76b8 P 7f1c581731e14b9fa50634b9a50957e1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } }
I20260811 15:08:21.790761 1023 raft_consensus.cc:374] T fcae63eac50048c4aed0b8753edf76b8 P 7f1c581731e14b9fa50634b9a50957e1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:21.790791 1023 raft_consensus.cc:729] T fcae63eac50048c4aed0b8753edf76b8 P 7f1c581731e14b9fa50634b9a50957e1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7f1c581731e14b9fa50634b9a50957e1, State: Initialized, Role: FOLLOWER
I20260811 15:08:21.790947 1023 consensus_queue.cc:260] T fcae63eac50048c4aed0b8753edf76b8 P 7f1c581731e14b9fa50634b9a50957e1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } }
I20260811 15:08:21.791221 1023 ts_tablet_manager.cc:1428] T fcae63eac50048c4aed0b8753edf76b8 P 7f1c581731e14b9fa50634b9a50957e1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:21.791477 1033 tablet_replica.cc:439] T fcae63eac50048c4aed0b8753edf76b8 P 7f1c581731e14b9fa50634b9a50957e1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } } }
I20260811 15:08:21.791640 1033 tablet_replica.cc:442] T fcae63eac50048c4aed0b8753edf76b8 P 7f1c581731e14b9fa50634b9a50957e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:22.049459 1035 raft_consensus.cc:482] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:08:22.049628 1035 raft_consensus.cc:504] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } }
I20260811 15:08:22.050854 1035 leader_election.cc:288] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7f1c581731e14b9fa50634b9a50957e1 (127.30.8.1:36389), a170158c4d8e456ba756a6bbd95120e1 (127.30.8.3:35451)
I20260811 15:08:22.055572 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:22.051168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:35451 (local address 127.30.8.2:39155)
0811 15:08:22.051523 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:22.051536 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:22.051668 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:22.051979 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:22.051982 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:22.051997 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:22.052115 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:22.052122 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.052880 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.052884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:22.054234 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:22.054246 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.055024 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.055028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:22.055167 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:22.055197 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 15:08:22.055305 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 15:08:22.055389 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"mutex_wait_us":27,"spinlock_wait_cycles":47872,"thread_start_us":104,"threads_started":1}
I20260811 15:08:22.055797 1021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:22.051631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:39155 (local address 127.30.8.3:35451)
0811 15:08:22.051782 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:22.051785 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:22.051797 (+ 12us) server_negotiation.cc:407] Connection header received
0811 15:08:22.051847 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:22.051849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:22.051887 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:22.051952 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:22.052238 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.052776 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.054354 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.054810 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.054944 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:22.055028 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:22.055380 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 15:08:22.055547 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 15:08:22.055757 1088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:22.051152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:36389 (local address 127.30.8.2:43191)
0811 15:08:22.051516 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:22.051533 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:22.051670 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:22.052026 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:22.052029 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:22.052045 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:22.052144 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:22.052150 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.053240 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.053244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:22.054386 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:22.054394 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.055390 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.055394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:22.055476 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:22.055497 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 15:08:22.055561 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 15:08:22.055625 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"mutex_wait_us":1,"spinlock_wait_cycles":9472,"thread_start_us":93,"threads_started":1}
I20260811 15:08:22.056044 1093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:22.051396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:43191 (local address 127.30.8.1:36389)
0811 15:08:22.051819 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:22.051822 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:22.051834 (+ 12us) server_negotiation.cc:407] Connection header received
0811 15:08:22.051880 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:22.051882 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:22.051920 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:22.051999 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:22.052272 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.053134 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.054492 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.055164 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.055270 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:22.055335 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:22.055827 (+ 492us) server_negotiation.cc:299] Negotiation successful
0811 15:08:22.055919 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":66,"threads_started":1}
I20260811 15:08:22.056480 424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcae63eac50048c4aed0b8753edf76b8" candidate_uuid: "f660d809fcb94aec94e3d9641c5193ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f1c581731e14b9fa50634b9a50957e1" is_pre_election: true
I20260811 15:08:22.056529 902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcae63eac50048c4aed0b8753edf76b8" candidate_uuid: "f660d809fcb94aec94e3d9641c5193ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a170158c4d8e456ba756a6bbd95120e1" is_pre_election: true
I20260811 15:08:22.056633 424 raft_consensus.cc:2454] T fcae63eac50048c4aed0b8753edf76b8 P 7f1c581731e14b9fa50634b9a50957e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f660d809fcb94aec94e3d9641c5193ec in term 0.
I20260811 15:08:22.056782 902 raft_consensus.cc:2454] T fcae63eac50048c4aed0b8753edf76b8 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f660d809fcb94aec94e3d9641c5193ec in term 0.
I20260811 15:08:22.057039 548 leader_election.cc:302] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f1c581731e14b9fa50634b9a50957e1, f660d809fcb94aec94e3d9641c5193ec; no voters:
I20260811 15:08:22.057340 1035 raft_consensus.cc:2781] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:08:22.057403 1035 raft_consensus.cc:482] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:08:22.057431 1035 raft_consensus.cc:3037] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:22.058454 1035 raft_consensus.cc:504] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } }
I20260811 15:08:22.058823 1035 leader_election.cc:288] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec [CANDIDATE]: Term 1 election: Requested vote from peers 7f1c581731e14b9fa50634b9a50957e1 (127.30.8.1:36389), a170158c4d8e456ba756a6bbd95120e1 (127.30.8.3:35451)
I20260811 15:08:22.059218 424 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcae63eac50048c4aed0b8753edf76b8" candidate_uuid: "f660d809fcb94aec94e3d9641c5193ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7f1c581731e14b9fa50634b9a50957e1"
I20260811 15:08:22.059226 902 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fcae63eac50048c4aed0b8753edf76b8" candidate_uuid: "f660d809fcb94aec94e3d9641c5193ec" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a170158c4d8e456ba756a6bbd95120e1"
I20260811 15:08:22.059327 902 raft_consensus.cc:3037] T fcae63eac50048c4aed0b8753edf76b8 P a170158c4d8e456ba756a6bbd95120e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:22.059337 424 raft_consensus.cc:3037] T fcae63eac50048c4aed0b8753edf76b8 P 7f1c581731e14b9fa50634b9a50957e1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:22.060290 902 raft_consensus.cc:2454] T fcae63eac50048c4aed0b8753edf76b8 P a170158c4d8e456ba756a6bbd95120e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f660d809fcb94aec94e3d9641c5193ec in term 1.
I20260811 15:08:22.060293 424 raft_consensus.cc:2454] T fcae63eac50048c4aed0b8753edf76b8 P 7f1c581731e14b9fa50634b9a50957e1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f660d809fcb94aec94e3d9641c5193ec in term 1.
I20260811 15:08:22.060560 548 leader_election.cc:302] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7f1c581731e14b9fa50634b9a50957e1, f660d809fcb94aec94e3d9641c5193ec; no voters:
I20260811 15:08:22.060743 1035 raft_consensus.cc:2781] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:22.060961 1035 raft_consensus.cc:686] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec [term 1 LEADER]: Becoming Leader. State: Replica: f660d809fcb94aec94e3d9641c5193ec, State: Running, Role: LEADER
I20260811 15:08:22.061224 1035 consensus_queue.cc:237] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } }
I20260811 15:08:22.062520 1094 tablet_replica.cc:439] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec: TxnStatusTablet state changed. Reason: New leader f660d809fcb94aec94e3d9641c5193ec. Latest consensus state: current_term: 1 leader_uuid: "f660d809fcb94aec94e3d9641c5193ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } } }
I20260811 15:08:22.062623 1094 tablet_replica.cc:442] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec: This TxnStatusTablet replica's current role is: LEADER
I20260811 15:08:22.062947 1097 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 15:08:22.063192 1097 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 15:08:22.063982 32611 catalog_manager.cc:5495] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec reported cstate change: term changed from 0 to 1, leader changed from <none> to f660d809fcb94aec94e3d9641c5193ec (127.30.8.2). New cstate: current_term: 1 leader_uuid: "f660d809fcb94aec94e3d9641c5193ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } health_report { overall_health: UNKNOWN } } }
I20260811 15:08:22.089453 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:22.085095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:38015 (local address 127.0.0.1:46468)
0811 15:08:22.085255 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:22.085271 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:22.085381 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:22.086138 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:22.086142 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:22.086153 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 15:08:22.086264 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:22.086269 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.087133 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.087137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:22.087755 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:22.087763 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.088213 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.088217 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:22.088306 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:22.088362 (+ 56us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 15:08:22.089117 (+ 755us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 15:08:22.089158 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 15:08:22.089222 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 15:08:22.089320 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 15:08:22.089473 1101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:22.085573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46468 (local address 127.30.8.2:38015)
0811 15:08:22.085846 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:22.085853 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:22.085865 (+ 12us) server_negotiation.cc:407] Connection header received
0811 15:08:22.085932 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:22.085935 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:22.085998 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:22.086100 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:22.086397 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.087024 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.087876 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.088111 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.088201 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:22.088534 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:22.089020 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:22.089128 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:22.089236 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 15:08:22.089296 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":117,"threads_started":1}
I20260811 15:08:22.092947 424 raft_consensus.cc:1261] T fcae63eac50048c4aed0b8753edf76b8 P 7f1c581731e14b9fa50634b9a50957e1 [term 1 FOLLOWER]: Refusing update from remote peer f660d809fcb94aec94e3d9641c5193ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:22.093112 902 raft_consensus.cc:1261] T fcae63eac50048c4aed0b8753edf76b8 P a170158c4d8e456ba756a6bbd95120e1 [term 1 FOLLOWER]: Refusing update from remote peer f660d809fcb94aec94e3d9641c5193ec: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:22.093482 1035 consensus_queue.cc:1035] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:22.093737 1094 consensus_queue.cc:1035] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:22.095417 1033 tablet_replica.cc:439] T fcae63eac50048c4aed0b8753edf76b8 P 7f1c581731e14b9fa50634b9a50957e1: TxnStatusTablet state changed. Reason: New leader f660d809fcb94aec94e3d9641c5193ec. Latest consensus state: current_term: 1 leader_uuid: "f660d809fcb94aec94e3d9641c5193ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } } }
I20260811 15:08:22.095546 1033 tablet_replica.cc:442] T fcae63eac50048c4aed0b8753edf76b8 P 7f1c581731e14b9fa50634b9a50957e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:22.097190 1034 tablet_replica.cc:439] T fcae63eac50048c4aed0b8753edf76b8 P a170158c4d8e456ba756a6bbd95120e1: TxnStatusTablet state changed. Reason: New leader f660d809fcb94aec94e3d9641c5193ec. Latest consensus state: current_term: 1 leader_uuid: "f660d809fcb94aec94e3d9641c5193ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } } }
I20260811 15:08:22.097312 1034 tablet_replica.cc:442] T fcae63eac50048c4aed0b8753edf76b8 P a170158c4d8e456ba756a6bbd95120e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:22.097539 1035 tablet_replica.cc:439] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f660d809fcb94aec94e3d9641c5193ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } } }
I20260811 15:08:22.097672 1035 tablet_replica.cc:442] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec: This TxnStatusTablet replica's current role is: LEADER
I20260811 15:08:22.098454 1044 mvcc.cc:205] Tried to move back new op lower bound from 7317343854967050240 to 7317343854845530112. Current Snapshot: MvccSnapshot[applied={T|T < 7317343854967050240}]
I20260811 15:08:22.098640 1033 tablet_replica.cc:439] T fcae63eac50048c4aed0b8753edf76b8 P 7f1c581731e14b9fa50634b9a50957e1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f660d809fcb94aec94e3d9641c5193ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } } }
I20260811 15:08:22.098734 1033 tablet_replica.cc:442] T fcae63eac50048c4aed0b8753edf76b8 P 7f1c581731e14b9fa50634b9a50957e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:22.100093 1094 tablet_replica.cc:439] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f660d809fcb94aec94e3d9641c5193ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } } }
I20260811 15:08:22.100210 1094 tablet_replica.cc:442] T fcae63eac50048c4aed0b8753edf76b8 P f660d809fcb94aec94e3d9641c5193ec: This TxnStatusTablet replica's current role is: LEADER
I20260811 15:08:22.102144 1034 tablet_replica.cc:439] T fcae63eac50048c4aed0b8753edf76b8 P a170158c4d8e456ba756a6bbd95120e1: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f660d809fcb94aec94e3d9641c5193ec" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 } } peers { permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 } } peers { permanent_uuid: "a170158c4d8e456ba756a6bbd95120e1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35451 } } }
I20260811 15:08:22.102267 1034 tablet_replica.cc:442] T fcae63eac50048c4aed0b8753edf76b8 P a170158c4d8e456ba756a6bbd95120e1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:22.113657 1109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:22.110154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:38015 (local address 127.30.8.3:56825)
0811 15:08:22.110451 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:22.110467 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:22.110586 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:22.110851 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:22.110854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:22.110862 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 15:08:22.111005 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:22.111011 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.111776 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.111780 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:22.112423 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:22.112432 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.112884 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.112888 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:22.112977 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:22.113028 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 15:08:22.113334 (+ 306us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 15:08:22.113340 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 15:08:22.113400 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 15:08:22.113498 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":88,"threads_started":1}
I20260811 15:08:22.113651 1101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:22.110301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:56825 (local address 127.30.8.2:38015)
0811 15:08:22.110463 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:22.110468 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:22.110522 (+ 54us) server_negotiation.cc:407] Connection header received
0811 15:08:22.110716 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:22.110719 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:22.110767 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:22.110829 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:22.111123 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.111666 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.112540 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.112773 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.112864 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:22.113169 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:22.113253 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:22.113312 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:22.113411 (+ 99us) server_negotiation.cc:299] Negotiation successful
0811 15:08:22.113487 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 15:08:22.121474 424 raft_consensus.cc:1261] T 906e7ec82992494b8a3c6d6cc29511b4 P 7f1c581731e14b9fa50634b9a50957e1 [term 1 FOLLOWER]: Refusing update from remote peer a170158c4d8e456ba756a6bbd95120e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:22.121511 658 raft_consensus.cc:1261] T 906e7ec82992494b8a3c6d6cc29511b4 P f660d809fcb94aec94e3d9641c5193ec [term 1 FOLLOWER]: Refusing update from remote peer a170158c4d8e456ba756a6bbd95120e1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:22.121891 1034 consensus_queue.cc:1035] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7f1c581731e14b9fa50634b9a50957e1" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:22.122083 1039 consensus_queue.cc:1035] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f660d809fcb94aec94e3d9641c5193ec" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 38015 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:22.149067 1021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:22.144823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:59883 (local address 127.30.8.3:35451)
0811 15:08:22.144942 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:22.144946 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:22.145164 (+ 218us) server_negotiation.cc:407] Connection header received
0811 15:08:22.145287 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:22.145289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:22.145337 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:22.145398 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:22.145704 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.146492 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.147830 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.148103 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.148203 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:22.148526 (+ 323us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:22.148623 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:22.148687 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:22.148837 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 15:08:22.148907 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 15:08:22.149536 1115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:22.144714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:35451 (local address 127.30.8.2:59883)
0811 15:08:22.145072 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:22.145088 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:22.145207 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:22.145417 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:22.145420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:22.145427 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 15:08:22.145566 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:22.145572 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.146650 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.146654 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:22.147712 (+ 1058us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:22.147724 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.148222 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.148226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:22.148323 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:22.148393 (+ 70us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 15:08:22.148713 (+ 320us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 15:08:22.148721 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 15:08:22.149281 (+ 560us) client_negotiation.cc:240] Negotiation successful
0811 15:08:22.149376 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":137,"threads_started":1}
I20260811 15:08:22.176848 32612 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:22.177057 32612 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49654:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:22.179299 32612 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a19dd1a8b19947ed957a34d45d46c83c: Sending DeleteTablet for 3 replicas of tablet 906e7ec82992494b8a3c6d6cc29511b4
I20260811 15:08:22.180038 869 tablet_service.cc:1514] Processing DeleteTablet for tablet 906e7ec82992494b8a3c6d6cc29511b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:22 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:52680
I20260811 15:08:22.180038 392 tablet_service.cc:1514] Processing DeleteTablet for tablet 906e7ec82992494b8a3c6d6cc29511b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:22 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:45856
I20260811 15:08:22.180073 612 tablet_service.cc:1514] Processing DeleteTablet for tablet 906e7ec82992494b8a3c6d6cc29511b4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:22 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:46458
I20260811 15:08:22.180454 1119 tablet_replica.cc:330] T 906e7ec82992494b8a3c6d6cc29511b4 P 7f1c581731e14b9fa50634b9a50957e1: stopping tablet replica
I20260811 15:08:22.180609 1119 raft_consensus.cc:2229] T 906e7ec82992494b8a3c6d6cc29511b4 P 7f1c581731e14b9fa50634b9a50957e1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:22.180742 1119 raft_consensus.cc:2258] T 906e7ec82992494b8a3c6d6cc29511b4 P 7f1c581731e14b9fa50634b9a50957e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:22.181483 1119 ts_tablet_manager.cc:1905] T 906e7ec82992494b8a3c6d6cc29511b4 P 7f1c581731e14b9fa50634b9a50957e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:22.181715 1120 tablet_replica.cc:330] T 906e7ec82992494b8a3c6d6cc29511b4 P f660d809fcb94aec94e3d9641c5193ec: stopping tablet replica
I20260811 15:08:22.181847 1120 raft_consensus.cc:2229] T 906e7ec82992494b8a3c6d6cc29511b4 P f660d809fcb94aec94e3d9641c5193ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:22.181978 1120 raft_consensus.cc:2258] T 906e7ec82992494b8a3c6d6cc29511b4 P f660d809fcb94aec94e3d9641c5193ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:22.182219 1118 tablet_replica.cc:330] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1: stopping tablet replica
I20260811 15:08:22.182356 1118 raft_consensus.cc:2229] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:08:22.182579 1118 raft_consensus.cc:2258] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:22.183250 1118 ts_tablet_manager.cc:1905] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:22.185940 1119 ts_tablet_manager.cc:1918] T 906e7ec82992494b8a3c6d6cc29511b4 P 7f1c581731e14b9fa50634b9a50957e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 15:08:22.186041 1119 log.cc:1196] T 906e7ec82992494b8a3c6d6cc29511b4 P 7f1c581731e14b9fa50634b9a50957e1: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-0/wal/wals/906e7ec82992494b8a3c6d6cc29511b4
I20260811 15:08:22.186304 1119 ts_tablet_manager.cc:1939] T 906e7ec82992494b8a3c6d6cc29511b4 P 7f1c581731e14b9fa50634b9a50957e1: Deleting consensus metadata
I20260811 15:08:22.187148 1120 ts_tablet_manager.cc:1905] T 906e7ec82992494b8a3c6d6cc29511b4 P f660d809fcb94aec94e3d9641c5193ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:22.187175 1118 ts_tablet_manager.cc:1918] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 15:08:22.187287 1118 log.cc:1196] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/906e7ec82992494b8a3c6d6cc29511b4
I20260811 15:08:22.187470 32593 catalog_manager.cc:4841] TS 7f1c581731e14b9fa50634b9a50957e1 (127.30.8.1:36389): tablet 906e7ec82992494b8a3c6d6cc29511b4 (table test-table [id=9514d744e7ee4d81ab0062d2cee20883]) successfully deleted
I20260811 15:08:22.187618 1118 ts_tablet_manager.cc:1939] T 906e7ec82992494b8a3c6d6cc29511b4 P a170158c4d8e456ba756a6bbd95120e1: Deleting consensus metadata
I20260811 15:08:22.188545 32594 catalog_manager.cc:4841] TS a170158c4d8e456ba756a6bbd95120e1 (127.30.8.3:35451): tablet 906e7ec82992494b8a3c6d6cc29511b4 (table test-table [id=9514d744e7ee4d81ab0062d2cee20883]) successfully deleted
I20260811 15:08:22.190443 1120 ts_tablet_manager.cc:1918] T 906e7ec82992494b8a3c6d6cc29511b4 P f660d809fcb94aec94e3d9641c5193ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 15:08:22.190552 1120 log.cc:1196] T 906e7ec82992494b8a3c6d6cc29511b4 P f660d809fcb94aec94e3d9641c5193ec: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/906e7ec82992494b8a3c6d6cc29511b4
I20260811 15:08:22.190829 1120 ts_tablet_manager.cc:1939] T 906e7ec82992494b8a3c6d6cc29511b4 P f660d809fcb94aec94e3d9641c5193ec: Deleting consensus metadata
I20260811 15:08:22.191783 32593 catalog_manager.cc:4841] TS f660d809fcb94aec94e3d9641c5193ec (127.30.8.2:38015): tablet 906e7ec82992494b8a3c6d6cc29511b4 (table test-table [id=9514d744e7ee4d81ab0062d2cee20883]) successfully deleted
Aug 11 15:08:22 dist-test-slave-1vmg krb5kdc[32542](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460901, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.8.1@KRBTEST.COM
I20260811 15:08:22.200050 1093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:22.181924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45862 (local address 127.30.8.1:36389)
0811 15:08:22.182050 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:22.182055 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:22.189117 (+ 7062us) server_negotiation.cc:407] Connection header received
0811 15:08:22.189289 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:22.189293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:22.189367 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:22.189440 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:22.190245 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.190915 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.191935 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.192344 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.192523 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:22.195706 (+ 3183us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:22.195724 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:22.195744 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:22.195784 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:22.197856 (+ 2072us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:22.198427 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:22.198434 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:22.198436 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:22.198545 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:22.198820 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:22.198823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:22.198825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:22.199257 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:22.199489 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:22.199685 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 15:08:22.199852 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260811 15:08:22.200642 496 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45862
I20260811 15:08:22.210355 32700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:22.202664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49666 (local address 127.30.8.62:43365)
0811 15:08:22.202807 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:22.202812 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:22.202829 (+ 17us) server_negotiation.cc:407] Connection header received
0811 15:08:22.202938 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:22.202941 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:22.202995 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:22.203084 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:22.203747 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.204551 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.205366 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:22.205714 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:22.205832 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:22.207182 (+ 1350us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:22.207187 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:22.207190 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:22.207218 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:22.208750 (+ 1532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:22.209235 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:22.209238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:22.209239 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:22.209314 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:22.209562 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:22.209564 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:22.209565 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:22.209740 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:22.209837 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:22.210117 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 15:08:22.210218 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
W20260811 15:08:22.211016 32612 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:49666
I20260811 15:08:22.212369 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 32747
I20260811 15:08:22.246888 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 515
I20260811 15:08:22.265388 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 753
I20260811 15:08:22.281580 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 32563
2026-08-11T15:08:22Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3317 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:08:22 dist-test-slave-1vmg krb5kdc[1132](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:08:22 dist-test-slave-1vmg krb5kdc[1132](info): set up 2 sockets
Aug 11 15:08:22 dist-test-slave-1vmg krb5kdc[1132](info): commencing operation
krb5kdc: starting...
W20260811 15:08:23.495430 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:08:23 dist-test-slave-1vmg krb5kdc[1132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:08:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:08:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:08:23.623947 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:45639
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:45639
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:23.738343 1147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:23.738657 1147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:23.738713 1147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:23.742753 1147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:08:23.742831 1147 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 15:08:23.742854 1147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:23.742878 1147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:23.742898 1147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:08:23.742951 1147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:08:23.748507 1147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:45639
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:45639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:23.749692 1147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:23.750722 1147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:23.759301 1153 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:23.759294 1157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:23.759339 1155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:23.759488 1147 server_base.cc:1034] running on GCE node
I20260811 15:08:23.760339 1147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:23.761508 1147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:23.762665 1147 hybrid_clock.cc:648] HybridClock initialized: now 1786460903762643 us; error 41 us; skew 500 ppm
Aug 11 15:08:23 dist-test-slave-1vmg krb5kdc[1132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460903, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:23.772184 1147 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:08:23.773250 1147 webserver.cc:460] Webserver started at http://127.30.8.62:40633/ using document root <none> and password file <none>
I20260811 15:08:23.773852 1147 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:23.773928 1147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:23.774166 1147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:23.776360 1147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "1e2eeb347cb845fa9536f56ef58c7365"
format_stamp: "Formatted at 2026-08-11 15:08:23 on dist-test-slave-1vmg"
server_key: "7ce41bd64819dcf28e9f8340afba74de"
server_key_iv: "86a37fe62d8f4eaf9427c928d9ff3ccb"
server_key_version: "encryptionkey@0"
I20260811 15:08:23.776923 1147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "1e2eeb347cb845fa9536f56ef58c7365"
format_stamp: "Formatted at 2026-08-11 15:08:23 on dist-test-slave-1vmg"
server_key: "7ce41bd64819dcf28e9f8340afba74de"
server_key_iv: "86a37fe62d8f4eaf9427c928d9ff3ccb"
server_key_version: "encryptionkey@0"
I20260811 15:08:23.780874 1147 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:08:23.783368 1168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:23.784578 1147 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:08:23.784709 1147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "1e2eeb347cb845fa9536f56ef58c7365"
format_stamp: "Formatted at 2026-08-11 15:08:23 on dist-test-slave-1vmg"
server_key: "7ce41bd64819dcf28e9f8340afba74de"
server_key_iv: "86a37fe62d8f4eaf9427c928d9ff3ccb"
server_key_version: "encryptionkey@0"
I20260811 15:08:23.784819 1147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:23.807348 1147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:23.810406 1147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:23.810597 1147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:23.818854 1147 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:45639
I20260811 15:08:23.818874 1269 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:45639 every 8 connection(s)
I20260811 15:08:23.819922 1147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:08:23.821158 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 1147
I20260811 15:08:23.821262 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:08:23.821516 30752 external_mini_cluster.cc:1428] Setting key 56ce31fc6233f6d8a4b5a96a85905ef4
I20260811 15:08:23.823184 1271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:23.830067 1271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365: Bootstrap starting.
Aug 11 15:08:23 dist-test-slave-1vmg krb5kdc[1132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:23.832935 1271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:23.834319 1271 log.cc:824] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:23.837122 1274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:23.822850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53294 (local address 127.30.8.62:45639)
0811 15:08:23.823297 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:23.823318 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:23.823359 (+ 41us) server_negotiation.cc:407] Connection header received
0811 15:08:23.824011 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:23.824027 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:23.824368 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:23.824668 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:23.825601 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:23.826621 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:23.827806 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:23.828299 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:23.828417 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:23.831302 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:23.831311 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:23.831330 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:23.831366 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:23.833367 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:23.833821 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:23.833828 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:23.833834 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:23.833984 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:23.834244 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:23.834247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:23.834249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:23.834843 (+ 594us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:23.835044 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:23.835653 (+ 609us) server_negotiation.cc:299] Negotiation successful
0811 15:08:23.836130 (+ 477us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":117,"threads_started":1}
I20260811 15:08:23.837247 1271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365: No bootstrap required, opened a new log
I20260811 15:08:23.841497 1271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2eeb347cb845fa9536f56ef58c7365" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45639 } }
I20260811 15:08:23.841845 1271 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:23.841889 1271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e2eeb347cb845fa9536f56ef58c7365, State: Initialized, Role: FOLLOWER
I20260811 15:08:23.842540 1271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2eeb347cb845fa9536f56ef58c7365" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45639 } }
I20260811 15:08:23.842751 1271 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:08:23.842816 1271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:08:23.842969 1271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:23.844491 1271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2eeb347cb845fa9536f56ef58c7365" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45639 } }
I20260811 15:08:23.844942 1271 leader_election.cc:302] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e2eeb347cb845fa9536f56ef58c7365; no voters:
I20260811 15:08:23.845392 1271 leader_election.cc:288] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:08:23.845567 1277 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:23.845927 1277 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365 [term 1 LEADER]: Becoming Leader. State: Replica: 1e2eeb347cb845fa9536f56ef58c7365, State: Running, Role: LEADER
I20260811 15:08:23.846374 1277 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2eeb347cb845fa9536f56ef58c7365" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45639 } }
I20260811 15:08:23.847286 1271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:08:23.848506 1280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e2eeb347cb845fa9536f56ef58c7365. Latest consensus state: current_term: 1 leader_uuid: "1e2eeb347cb845fa9536f56ef58c7365" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2eeb347cb845fa9536f56ef58c7365" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45639 } } }
I20260811 15:08:23.848685 1280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:23.849020 1279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e2eeb347cb845fa9536f56ef58c7365" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e2eeb347cb845fa9536f56ef58c7365" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45639 } } }
I20260811 15:08:23.849130 1279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:23.849151 1290 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:08:23.852483 1290 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:08:23.859781 1290 catalog_manager.cc:1347] Generated new cluster ID: 6570325d336a42fca3d5a9344da5f214
I20260811 15:08:23.859877 1290 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:08:23.877656 1290 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:08:23.878890 1290 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:08:23.890157 1290 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365: Generated new TSK 0
I20260811 15:08:23.891032 1290 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
I20260811 15:08:23.934139 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45639
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:24.060600 1313 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:24.060925 1313 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:24.061002 1313 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:24.065323 1313 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:24.065411 1313 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:24.065470 1313 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 15:08:24.065492 1313 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 15:08:24.065549 1313 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:08:24.071245 1313 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.8.62:45639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1313
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:24.072546 1313 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:24.073633 1313 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:24.084733 1326 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:24.084733 1321 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:24.084771 1323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:24.085001 1313 server_base.cc:1034] running on GCE node
I20260811 15:08:24.085628 1313 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:24.086380 1313 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:24.087564 1313 hybrid_clock.cc:648] HybridClock initialized: now 1786460904087546 us; error 32 us; skew 500 ppm
Aug 11 15:08:24 dist-test-slave-1vmg krb5kdc[1132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460904, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:24.090564 1313 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:08:24.091681 1313 webserver.cc:460] Webserver started at http://127.30.8.1:34443/ using document root <none> and password file <none>
I20260811 15:08:24.092281 1313 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:24.092355 1313 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:24.092566 1313 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:24.094761 1313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "047b159c89f543b185fc7da705f3eac5"
format_stamp: "Formatted at 2026-08-11 15:08:24 on dist-test-slave-1vmg"
server_key: "4488f811f33a0f18eea037b87bc91328"
server_key_iv: "e713d8f42cd674f735e31a738420cfca"
server_key_version: "encryptionkey@0"
I20260811 15:08:24.095394 1313 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "047b159c89f543b185fc7da705f3eac5"
format_stamp: "Formatted at 2026-08-11 15:08:24 on dist-test-slave-1vmg"
server_key: "4488f811f33a0f18eea037b87bc91328"
server_key_iv: "e713d8f42cd674f735e31a738420cfca"
server_key_version: "encryptionkey@0"
I20260811 15:08:24.099409 1313 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:08:24.102152 1343 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:24.103500 1313 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:08:24.103626 1313 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "047b159c89f543b185fc7da705f3eac5"
format_stamp: "Formatted at 2026-08-11 15:08:24 on dist-test-slave-1vmg"
server_key: "4488f811f33a0f18eea037b87bc91328"
server_key_iv: "e713d8f42cd674f735e31a738420cfca"
server_key_version: "encryptionkey@0"
I20260811 15:08:24.103745 1313 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:24.124768 1313 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:24.128422 1313 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:24.128625 1313 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:24.130223 1313 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:24.130280 1313 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:24.130336 1313 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:24.130365 1313 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 15:08:24 dist-test-slave-1vmg krb5kdc[1132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460904, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:24.141254 1313 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:33327
I20260811 15:08:24.141393 1541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:33327 every 8 connection(s)
I20260811 15:08:24.142302 1313 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:08:24.143481 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 1313
I20260811 15:08:24.143586 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:08:24.143859 30752 external_mini_cluster.cc:1428] Setting key 6ea2d23bd9102532c48a1d9251e33902
I20260811 15:08:24.144487 1274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.131121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:48319 (local address 127.30.8.62:45639)
0811 15:08:24.131247 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:24.131251 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:24.132364 (+ 1113us) server_negotiation.cc:407] Connection header received
0811 15:08:24.133453 (+ 1089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:24.133457 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:24.133521 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:24.133597 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:24.135141 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.135703 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.136922 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.137169 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.137276 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.140438 (+ 3162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:24.140442 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:24.140444 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:24.140472 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:24.142016 (+ 1544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:24.142648 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:24.142652 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:24.142654 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:24.142766 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:24.143241 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:24.143244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:24.143246 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:24.143512 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:24.143615 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:24.144150 (+ 535us) server_negotiation.cc:299] Negotiation successful
0811 15:08:24.144318 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 15:08:24.145411 1359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.131653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45639 (local address 127.30.8.1:48319)
0811 15:08:24.132194 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:24.132238 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:24.133188 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:24.134031 (+ 843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:24.134039 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:24.134386 (+ 347us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:24.134980 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.134996 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.135834 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.135838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.136792 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.136803 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.137347 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.137352 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.137580 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.138340 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:24.138357 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:24.140244 (+ 1887us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:24.142148 (+ 1904us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:24.142154 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:24.142169 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:24.142543 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:24.142876 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:24.142879 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:24.142881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:24.143104 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:24.143616 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:24.143623 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:24.143889 (+ 266us) client_negotiation.cc:769] Sending connection context
0811 15:08:24.144077 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 15:08:24.144423 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":144,"threads_started":1}
I20260811 15:08:24.146721 1542 heartbeater.cc:347] Connected to a master server at 127.30.8.62:45639
I20260811 15:08:24.146899 1542 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:24.147712 1542 heartbeater.cc:511] Master 127.30.8.62:45639 requested a full tablet report, sending...
I20260811 15:08:24.149572 1194 ts_manager.cc:194] Registered new tserver with Master: 047b159c89f543b185fc7da705f3eac5 (127.30.8.1:33327)
I20260811 15:08:24.151166 1194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:48319
WARNING: no policy specified for kudu/127.30.8.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.2@KRBTEST.COM" created.
I20260811 15:08:24.161865 1274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.149302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:53569 (local address 127.30.8.62:45639)
0811 15:08:24.149419 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:24.149424 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:24.155319 (+ 5895us) server_negotiation.cc:407] Connection header received
0811 15:08:24.155443 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:24.155446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:24.155490 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:24.155558 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:24.156028 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.156617 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.157451 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.157678 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.157767 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.158978 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:24.158981 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:24.158983 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:24.159006 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:24.160303 (+ 1297us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:24.160705 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:24.160708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:24.160710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:24.160785 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:24.161036 (+ 251us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:24.161039 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:24.161041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:24.161229 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:24.161318 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:24.161569 (+ 251us) server_negotiation.cc:299] Negotiation successful
0811 15:08:24.161701 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 15:08:24.161870 1551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.149254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45639 (local address 127.30.8.1:53569)
0811 15:08:24.155212 (+ 5958us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:24.155229 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:24.155355 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:24.155589 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:24.155592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:24.155804 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:24.155913 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.155919 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.156752 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.156756 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.157337 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.157346 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.157786 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.157790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.157886 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.158399 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:24.158412 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:24.158802 (+ 390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:24.160412 (+ 1610us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:24.160415 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:24.160417 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:24.160613 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:24.160867 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:24.160870 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:24.160872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:24.160954 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:24.161321 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:24.161324 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:24.161419 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 15:08:24.161506 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 15:08:24.161666 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5836,"thread_start_us":110,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
I20260811 15:08:24.178267 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45639
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:24.299628 1553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:24.299971 1553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:24.300042 1553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:24.304263 1553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:24.304352 1553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:24.304410 1553 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 15:08:24.304435 1553 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 15:08:24.304493 1553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:08:24.310434 1553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.8.62:45639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:24.311677 1553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:24.312747 1553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:24.323114 1561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:24.323262 1553 server_base.cc:1034] running on GCE node
W20260811 15:08:24.323164 1562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:24.323110 1564 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:24.323848 1553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:24.324573 1553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:24.325744 1553 hybrid_clock.cc:648] HybridClock initialized: now 1786460904325727 us; error 30 us; skew 500 ppm
Aug 11 15:08:24 dist-test-slave-1vmg krb5kdc[1132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460904, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:24.328637 1553 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:08:24.329664 1553 webserver.cc:460] Webserver started at http://127.30.8.2:42033/ using document root <none> and password file <none>
I20260811 15:08:24.330276 1553 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:24.330349 1553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:24.330569 1553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:24.332804 1553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "a5680e80a3cf4cfa8e872f64f6189066"
format_stamp: "Formatted at 2026-08-11 15:08:24 on dist-test-slave-1vmg"
server_key: "52fe9c7e77b9cb1b0f2439ca1d784a2c"
server_key_iv: "4e3e9d0b55bb0d8cf79fcdc7d1b12ddd"
server_key_version: "encryptionkey@0"
I20260811 15:08:24.333392 1553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "a5680e80a3cf4cfa8e872f64f6189066"
format_stamp: "Formatted at 2026-08-11 15:08:24 on dist-test-slave-1vmg"
server_key: "52fe9c7e77b9cb1b0f2439ca1d784a2c"
server_key_iv: "4e3e9d0b55bb0d8cf79fcdc7d1b12ddd"
server_key_version: "encryptionkey@0"
I20260811 15:08:24.337769 1553 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 15:08:24.341060 1575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:24.342669 1553 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 15:08:24.342864 1553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "a5680e80a3cf4cfa8e872f64f6189066"
format_stamp: "Formatted at 2026-08-11 15:08:24 on dist-test-slave-1vmg"
server_key: "52fe9c7e77b9cb1b0f2439ca1d784a2c"
server_key_iv: "4e3e9d0b55bb0d8cf79fcdc7d1b12ddd"
server_key_version: "encryptionkey@0"
I20260811 15:08:24.343048 1553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:24.357620 1553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:24.360821 1553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:24.361027 1553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:24.362629 1553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:24.362687 1553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:24.362746 1553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:24.362775 1553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 15:08:24 dist-test-slave-1vmg krb5kdc[1132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460904, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:24.373508 1553 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:44741
I20260811 15:08:24.373567 1773 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:44741 every 8 connection(s)
I20260811 15:08:24.374504 1553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:08:24.376014 1274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.363390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:42865 (local address 127.30.8.62:45639)
0811 15:08:24.363557 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:24.363562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:24.364885 (+ 1323us) server_negotiation.cc:407] Connection header received
0811 15:08:24.366076 (+ 1191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:24.366080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:24.366141 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:24.366227 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:24.367519 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.368129 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.369294 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.369509 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.369599 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.372337 (+ 2738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:24.372342 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:24.372345 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:24.372376 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:24.373834 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:24.374339 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:24.374343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:24.374345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:24.374424 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:24.374774 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:24.374777 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:24.374779 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:24.375045 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:24.375139 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:24.375765 (+ 626us) server_negotiation.cc:299] Negotiation successful
0811 15:08:24.375885 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 15:08:24.376335 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 1553
I20260811 15:08:24.376446 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:08:24.376726 30752 external_mini_cluster.cc:1428] Setting key 78d4b6545d93e131250e13e037526006
I20260811 15:08:24.376973 1592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.364214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45639 (local address 127.30.8.2:42865)
0811 15:08:24.364696 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:24.364759 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:24.365806 (+ 1047us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:24.366408 (+ 602us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:24.366415 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:24.366818 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:24.367369 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.367383 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.368251 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.368255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.369155 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.369167 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.369670 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.369674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.369881 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.370534 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:24.370552 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:24.372138 (+ 1586us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:24.373948 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:24.373954 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:24.373970 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:24.374240 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:24.374508 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:24.374511 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:24.374513 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:24.374678 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:24.375190 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:24.375197 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:24.375497 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 15:08:24.375686 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 15:08:24.375969 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":116,"threads_started":1}
I20260811 15:08:24.380256 1774 heartbeater.cc:347] Connected to a master server at 127.30.8.62:45639
I20260811 15:08:24.380431 1774 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:24.380959 1774 heartbeater.cc:511] Master 127.30.8.62:45639 requested a full tablet report, sending...
I20260811 15:08:24.382103 1194 ts_manager.cc:194] Registered new tserver with Master: a5680e80a3cf4cfa8e872f64f6189066 (127.30.8.2:44741)
I20260811 15:08:24.382809 1194 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:42865
WARNING: no policy specified for kudu/127.30.8.3@KRBTEST.COM; defaulting to no policy
I20260811 15:08:24.389442 1785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.380554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45639 (local address 127.30.8.2:44841)
0811 15:08:24.381609 (+ 1055us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:24.381622 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:24.381747 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:24.382016 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:24.382019 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:24.382260 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:24.382400 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.382407 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.383372 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.383376 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.384257 (+ 881us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.384271 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.384837 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.384841 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.384940 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.385457 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:24.385474 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:24.385937 (+ 463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:24.387527 (+ 1590us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:24.387531 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:24.387533 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:24.387788 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:24.388055 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:24.388058 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:24.388060 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:24.388166 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:24.388771 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:24.388775 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:24.388994 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 15:08:24.389089 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 15:08:24.389246 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":979,"thread_start_us":109,"threads_started":1}
I20260811 15:08:24.389487 1274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.380547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:44841 (local address 127.30.8.62:45639)
0811 15:08:24.380674 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:24.380679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:24.381691 (+ 1012us) server_negotiation.cc:407] Connection header received
0811 15:08:24.381845 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:24.381848 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:24.381900 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:24.381977 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:24.382523 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.383227 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.384393 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.384695 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.384806 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.386074 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:24.386078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:24.386080 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:24.386102 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:24.387411 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:24.387879 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:24.387882 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:24.387885 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:24.387962 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:24.388290 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:24.388293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:24.388295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:24.388528 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:24.388610 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:24.389192 (+ 582us) server_negotiation.cc:299] Negotiation successful
0811 15:08:24.389327 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
I20260811 15:08:24.408103 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45639
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:33551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:24.528152 1792 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:24.528470 1792 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:24.528546 1792 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:24.532783 1792 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:24.532873 1792 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:24.532935 1792 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 15:08:24.532960 1792 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 15:08:24.533020 1792 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:08:24.538786 1792 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:33551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.8.62:45639
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1792
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:24.540102 1792 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:24.541191 1792 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:24.551908 1802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:24.551918 1799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:24.551908 1800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:24.552309 1792 server_base.cc:1034] running on GCE node
I20260811 15:08:24.552821 1792 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:24.553545 1792 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:24.554724 1792 hybrid_clock.cc:648] HybridClock initialized: now 1786460904554729 us; error 61 us; skew 500 ppm
Aug 11 15:08:24 dist-test-slave-1vmg krb5kdc[1132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460904, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:24.558032 1792 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:08:24.559221 1792 webserver.cc:460] Webserver started at http://127.30.8.3:37869/ using document root <none> and password file <none>
I20260811 15:08:24.559865 1792 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:24.559947 1792 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:24.560194 1792 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:24.562439 1792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3"
format_stamp: "Formatted at 2026-08-11 15:08:24 on dist-test-slave-1vmg"
server_key: "9324ed8c77cd7d49d32b7fe80b9a2b6c"
server_key_iv: "176a8517a28fe7df83e45bbdf2f7eb28"
server_key_version: "encryptionkey@0"
I20260811 15:08:24.563127 1792 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3"
format_stamp: "Formatted at 2026-08-11 15:08:24 on dist-test-slave-1vmg"
server_key: "9324ed8c77cd7d49d32b7fe80b9a2b6c"
server_key_iv: "176a8517a28fe7df83e45bbdf2f7eb28"
server_key_version: "encryptionkey@0"
I20260811 15:08:24.567538 1792 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:08:24.570333 1815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:24.571722 1792 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:08:24.571864 1792 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3"
format_stamp: "Formatted at 2026-08-11 15:08:24 on dist-test-slave-1vmg"
server_key: "9324ed8c77cd7d49d32b7fe80b9a2b6c"
server_key_iv: "176a8517a28fe7df83e45bbdf2f7eb28"
server_key_version: "encryptionkey@0"
I20260811 15:08:24.571990 1792 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:24.619977 1792 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:24.623673 1792 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:24.623903 1792 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:24.625651 1792 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:24.625718 1792 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:24.625782 1792 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:24.625811 1792 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 15:08:24 dist-test-slave-1vmg krb5kdc[1132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460904, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:24.638123 1792 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:33441
I20260811 15:08:24.638166 2009 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:33441 every 8 connection(s)
I20260811 15:08:24.639314 1792 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:08:24.641283 1274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.626425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:46479 (local address 127.30.8.62:45639)
0811 15:08:24.626589 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:24.626595 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:24.628084 (+ 1489us) server_negotiation.cc:407] Connection header received
0811 15:08:24.629410 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:24.629414 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:24.629482 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:24.629581 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:24.631389 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.632003 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.633754 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.634043 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.634150 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.637104 (+ 2954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:24.637109 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:24.637112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:24.637146 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:24.638743 (+ 1597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:24.639418 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:24.639421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:24.639423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:24.639512 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:24.639962 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:24.639967 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:24.639969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:24.640268 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:24.640379 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:24.640943 (+ 564us) server_negotiation.cc:299] Negotiation successful
0811 15:08:24.641100 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 15:08:24.642086 1831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.627329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45639 (local address 127.30.8.3:46479)
0811 15:08:24.627867 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:24.627935 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:24.629140 (+ 1205us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:24.629790 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:24.629801 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:24.630364 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:24.631192 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.631218 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.632127 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.632131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.633613 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.633628 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.634207 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.634211 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.634405 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.635166 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:24.635183 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:24.636898 (+ 1715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:24.638871 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:24.638877 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:24.638893 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:24.639296 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:24.639605 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:24.639608 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:24.639610 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:24.639826 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:24.640390 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:24.640398 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:24.640672 (+ 274us) client_negotiation.cc:769] Sending connection context
0811 15:08:24.640860 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 15:08:24.641220 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":123,"threads_started":1}
I20260811 15:08:24.643347 2010 heartbeater.cc:347] Connected to a master server at 127.30.8.62:45639
I20260811 15:08:24.643507 2010 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:24.644116 2010 heartbeater.cc:511] Master 127.30.8.62:45639 requested a full tablet report, sending...
I20260811 15:08:24.645329 1193 ts_manager.cc:194] Registered new tserver with Master: b6d0487c1f6145f0aa0e09ed071cb3c3 (127.30.8.3:33441)
I20260811 15:08:24.646152 1193 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:46479
I20260811 15:08:24.646616 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 1792
I20260811 15:08:24.646775 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:08:24.647212 30752 external_mini_cluster.cc:1428] Setting key b90ec7a65de75763f90155c221b00146
I20260811 15:08:24.649557 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 15:08:24.653836 1274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.645979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:42389 (local address 127.30.8.62:45639)
0811 15:08:24.646099 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:24.646103 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:24.646229 (+ 126us) server_negotiation.cc:407] Connection header received
0811 15:08:24.646344 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:24.646347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:24.646389 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:24.646456 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:24.647189 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.647843 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.648980 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.649310 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.649408 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.650682 (+ 1274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:24.650686 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:24.650688 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:24.650714 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:24.652137 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:24.652603 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:24.652606 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:24.652607 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:24.652685 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:24.653018 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:24.653021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:24.653022 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:24.653245 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:24.653332 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:24.653602 (+ 270us) server_negotiation.cc:299] Negotiation successful
0811 15:08:24.653703 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 15:08:24.653923 2019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.645808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45639 (local address 127.30.8.3:42389)
0811 15:08:24.646147 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:24.646161 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:24.646261 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:24.646635 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:24.646637 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:24.646876 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:24.647054 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.647061 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.648009 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.648013 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.648864 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.648875 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.649424 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.649427 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.649527 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.650042 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:24.650057 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:24.650530 (+ 473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:24.652253 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:24.652257 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:24.652259 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:24.652512 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:24.652762 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:24.652765 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:24.652767 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:24.652889 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:24.653341 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:24.653344 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:24.653453 (+ 109us) client_negotiation.cc:769] Sending connection context
0811 15:08:24.653558 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 15:08:24.653717 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":149,"threads_started":1}
Aug 11 15:08:24 dist-test-slave-1vmg krb5kdc[1132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 15:08:24 dist-test-slave-1vmg krb5kdc[1132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460904, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:24.678799 1274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.670253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52704 (local address 127.30.8.62:45639)
0811 15:08:24.670376 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:24.670380 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:24.670392 (+ 12us) server_negotiation.cc:407] Connection header received
0811 15:08:24.670435 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:24.670437 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:24.670476 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:24.670546 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:24.671332 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.671860 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.672674 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.672891 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.672979 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.675415 (+ 2436us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:24.675419 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:24.675421 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:24.675449 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:24.677033 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:24.677485 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:24.677488 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:24.677489 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:24.677611 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:24.677897 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:24.677900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:24.677902 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:24.678107 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:24.678236 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:24.678481 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 15:08:24.678615 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 15:08:24.681797 1193 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:08:24.685146 1193 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:08:24.700368 2038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.694023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:44741 (local address 127.0.0.1:55802)
0811 15:08:24.694872 (+ 849us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:24.694954 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:24.695120 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:24.695880 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:24.695887 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:24.695910 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:24.696082 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.696090 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.697716 (+ 1626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.697719 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.698778 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.698789 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.699846 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.699849 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.699932 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.699966 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 15:08:24.700052 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 15:08:24.700178 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":674,"mutex_wait_us":59,"thread_start_us":84,"threads_started":1}
I20260811 15:08:24.700376 2040 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.694161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55802 (local address 127.30.8.2:44741)
0811 15:08:24.694582 (+ 421us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:24.694595 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:24.695045 (+ 450us) server_negotiation.cc:407] Connection header received
0811 15:08:24.695298 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:24.695304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:24.695533 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:24.695669 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:24.696185 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.697450 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.698903 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.699711 (+ 808us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.699848 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.700036 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:24.700098 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 15:08:24.700203 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":155,"threads_started":1}
I20260811 15:08:24.700513 2039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.694151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:33327 (local address 127.0.0.1:43890)
0811 15:08:24.694936 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:24.694957 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:24.695095 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:24.696015 (+ 920us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:24.696018 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:24.696036 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:24.696223 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.696230 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.697609 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.697612 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.699115 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.699127 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.700161 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.700164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.700255 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.700279 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 15:08:24.700348 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 15:08:24.700410 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":699,"mutex_wait_us":192,"thread_start_us":115,"threads_started":1}
I20260811 15:08:24.701793 2041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.694798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:33441 (local address 127.0.0.1:49256)
0811 15:08:24.695648 (+ 850us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:24.695659 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:24.695742 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:24.696393 (+ 651us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:24.696395 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:24.696414 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:24.696515 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.696522 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.697492 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.697496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.699115 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.699127 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.701396 (+ 2269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.701400 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.701500 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.701524 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 15:08:24.701574 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 15:08:24.701643 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"mutex_wait_us":88,"thread_start_us":124,"threads_started":1}
I20260811 15:08:24.702456 2043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.695313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49256 (local address 127.30.8.3:33441)
0811 15:08:24.695947 (+ 634us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:24.695959 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:24.695981 (+ 22us) server_negotiation.cc:407] Connection header received
0811 15:08:24.696052 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:24.696057 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:24.696234 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:24.696412 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:24.696625 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.697352 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.700344 (+ 2992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.701255 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.701994 (+ 739us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.702169 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:24.702223 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 15:08:24.702312 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":107,"threads_started":1}
I20260811 15:08:24.703094 2042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.695276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43890 (local address 127.30.8.1:33327)
0811 15:08:24.695587 (+ 311us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:24.695595 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:24.695616 (+ 21us) server_negotiation.cc:407] Connection header received
0811 15:08:24.695686 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:24.695690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:24.695876 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:24.696014 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:24.696354 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.697440 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.699254 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.700028 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.700144 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.700273 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:24.701255 (+ 982us) server_negotiation.cc:299] Negotiation successful
0811 15:08:24.702986 (+ 1731us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":106,"threads_started":1}
I20260811 15:08:24.704106 1664 tablet_service.cc:1467] Processing CreateTablet for tablet 992ef160a1cf445a905670b00de55fc7 (DEFAULT_TABLE table=test-table [id=cf88fe8390634e0b9f80707f2509effe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:24.705402 1664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 992ef160a1cf445a905670b00de55fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:24.705363 1897 tablet_service.cc:1467] Processing CreateTablet for tablet 992ef160a1cf445a905670b00de55fc7 (DEFAULT_TABLE table=test-table [id=cf88fe8390634e0b9f80707f2509effe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:24.705397 1425 tablet_service.cc:1467] Processing CreateTablet for tablet 992ef160a1cf445a905670b00de55fc7 (DEFAULT_TABLE table=test-table [id=cf88fe8390634e0b9f80707f2509effe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:24.706389 1425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 992ef160a1cf445a905670b00de55fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:24.706416 1897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 992ef160a1cf445a905670b00de55fc7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:24.712747 2044 tablet_bootstrap.cc:492] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5: Bootstrap starting.
I20260811 15:08:24.714505 2049 tablet_bootstrap.cc:492] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3: Bootstrap starting.
I20260811 15:08:24.715127 2044 tablet_bootstrap.cc:654] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:24.715269 2048 tablet_bootstrap.cc:492] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066: Bootstrap starting.
I20260811 15:08:24.715984 2044 log.cc:824] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:24.716684 2049 tablet_bootstrap.cc:654] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:24.717430 2048 tablet_bootstrap.cc:654] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:24.717564 2049 log.cc:824] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:24.718210 2048 log.cc:824] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:24.718538 2044 tablet_bootstrap.cc:492] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5: No bootstrap required, opened a new log
I20260811 15:08:24.718779 2044 ts_tablet_manager.cc:1397] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 15:08:24.719719 2049 tablet_bootstrap.cc:492] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3: No bootstrap required, opened a new log
I20260811 15:08:24.719942 2049 ts_tablet_manager.cc:1397] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 15:08:24.720250 2048 tablet_bootstrap.cc:492] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066: No bootstrap required, opened a new log
I20260811 15:08:24.720432 2048 ts_tablet_manager.cc:1397] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 15:08:24.722496 2044 raft_consensus.cc:348] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:24.722752 2044 raft_consensus.cc:374] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:24.722808 2044 raft_consensus.cc:729] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 047b159c89f543b185fc7da705f3eac5, State: Initialized, Role: FOLLOWER
I20260811 15:08:24.723316 2044 consensus_queue.cc:260] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:24.723927 2049 raft_consensus.cc:348] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:24.724201 2049 raft_consensus.cc:374] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:24.724263 2049 raft_consensus.cc:729] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6d0487c1f6145f0aa0e09ed071cb3c3, State: Initialized, Role: FOLLOWER
I20260811 15:08:24.724193 2048 raft_consensus.cc:348] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:24.724431 2048 raft_consensus.cc:374] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:24.724458 2044 ts_tablet_manager.cc:1428] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 15:08:24.724507 2048 raft_consensus.cc:729] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5680e80a3cf4cfa8e872f64f6189066, State: Initialized, Role: FOLLOWER
I20260811 15:08:24.724709 2049 consensus_queue.cc:260] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:24.724875 1542 heartbeater.cc:503] Master 127.30.8.62:45639 was elected leader, sending a full tablet report...
I20260811 15:08:24.724951 2048 consensus_queue.cc:260] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:24.725395 2010 heartbeater.cc:503] Master 127.30.8.62:45639 was elected leader, sending a full tablet report...
I20260811 15:08:24.725773 2049 ts_tablet_manager.cc:1428] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 15:08:24.725777 2048 ts_tablet_manager.cc:1428] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 15:08:24.726083 1774 heartbeater.cc:503] Master 127.30.8.62:45639 was elected leader, sending a full tablet report...
W20260811 15:08:24.875869 1775 tablet.cc:2369] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 15:08:24.890427 2011 tablet.cc:2369] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 15:08:24.893725 1543 tablet.cc:2369] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:08:24.936537 2058 raft_consensus.cc:482] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:08:24.936786 2058 raft_consensus.cc:504] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:24.938063 2058 leader_election.cc:288] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 047b159c89f543b185fc7da705f3eac5 (127.30.8.1:33327), b6d0487c1f6145f0aa0e09ed071cb3c3 (127.30.8.3:33441)
I20260811 15:08:24.942237 2042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.938575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:36271 (local address 127.30.8.1:33327)
0811 15:08:24.938763 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:24.938769 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:24.938906 (+ 137us) server_negotiation.cc:407] Connection header received
0811 15:08:24.939106 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:24.939109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:24.939161 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:24.939240 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:24.939531 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.940088 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.941234 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.941665 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.941765 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.941840 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:24.942005 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 15:08:24.942081 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 15:08:24.942253 2060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.938332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:33327 (local address 127.30.8.2:36271)
0811 15:08:24.938779 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:24.938796 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:24.938985 (+ 189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:24.939269 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:24.939272 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:24.939297 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:24.939420 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.939426 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.940200 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.940204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.941098 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.941108 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.941786 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.941789 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.941885 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.941911 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 15:08:24.941981 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 15:08:24.942079 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":107,"threads_started":1}
I20260811 15:08:24.942405 2043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.938679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:60609 (local address 127.30.8.3:33441)
0811 15:08:24.938802 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:24.938807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:24.939131 (+ 324us) server_negotiation.cc:407] Connection header received
0811 15:08:24.939302 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:24.939305 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:24.939361 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:24.939430 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:24.939720 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.940310 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.941361 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.941793 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.941892 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.941973 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:24.942159 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 15:08:24.942245 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 15:08:24.942628 2063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:24.938561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:33441 (local address 127.30.8.2:60609)
0811 15:08:24.939012 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:24.939027 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:24.939176 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:24.939454 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:24.939457 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:24.939475 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:24.939610 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.939616 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.940433 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.940436 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.941220 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:24.941229 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:24.941919 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:24.941923 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:24.942036 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:24.942062 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 15:08:24.942131 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 15:08:24.942222 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":97,"threads_started":1}
I20260811 15:08:24.943233 1460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "992ef160a1cf445a905670b00de55fc7" candidate_uuid: "a5680e80a3cf4cfa8e872f64f6189066" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047b159c89f543b185fc7da705f3eac5" is_pre_election: true
I20260811 15:08:24.943452 1929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "992ef160a1cf445a905670b00de55fc7" candidate_uuid: "a5680e80a3cf4cfa8e872f64f6189066" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" is_pre_election: true
I20260811 15:08:24.943728 1929 raft_consensus.cc:2454] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5680e80a3cf4cfa8e872f64f6189066 in term 0.
I20260811 15:08:24.943732 1460 raft_consensus.cc:2454] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a5680e80a3cf4cfa8e872f64f6189066 in term 0.
I20260811 15:08:24.944411 1582 leader_election.cc:302] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047b159c89f543b185fc7da705f3eac5, a5680e80a3cf4cfa8e872f64f6189066; no voters:
I20260811 15:08:24.944787 2058 raft_consensus.cc:2781] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:08:24.944928 2058 raft_consensus.cc:482] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:08:24.944983 2058 raft_consensus.cc:3037] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:24.946154 2058 raft_consensus.cc:504] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:24.946795 2058 leader_election.cc:288] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [CANDIDATE]: Term 1 election: Requested vote from peers 047b159c89f543b185fc7da705f3eac5 (127.30.8.1:33327), b6d0487c1f6145f0aa0e09ed071cb3c3 (127.30.8.3:33441)
I20260811 15:08:24.947355 1929 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "992ef160a1cf445a905670b00de55fc7" candidate_uuid: "a5680e80a3cf4cfa8e872f64f6189066" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3"
I20260811 15:08:24.947359 1460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "992ef160a1cf445a905670b00de55fc7" candidate_uuid: "a5680e80a3cf4cfa8e872f64f6189066" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047b159c89f543b185fc7da705f3eac5"
I20260811 15:08:24.947562 1460 raft_consensus.cc:3037] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:24.947563 1929 raft_consensus.cc:3037] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:24.949120 1460 raft_consensus.cc:2454] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5680e80a3cf4cfa8e872f64f6189066 in term 1.
I20260811 15:08:24.949573 1929 raft_consensus.cc:2454] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a5680e80a3cf4cfa8e872f64f6189066 in term 1.
I20260811 15:08:24.949558 1582 leader_election.cc:302] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047b159c89f543b185fc7da705f3eac5, a5680e80a3cf4cfa8e872f64f6189066; no voters:
I20260811 15:08:24.949781 2058 raft_consensus.cc:2781] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:24.950042 2058 raft_consensus.cc:686] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [term 1 LEADER]: Becoming Leader. State: Replica: a5680e80a3cf4cfa8e872f64f6189066, State: Running, Role: LEADER
I20260811 15:08:24.950418 2058 consensus_queue.cc:237] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:24.953728 1193 catalog_manager.cc:5495] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 reported cstate change: term changed from 0 to 1, leader changed from <none> to a5680e80a3cf4cfa8e872f64f6189066 (127.30.8.2). New cstate: current_term: 1 leader_uuid: "a5680e80a3cf4cfa8e872f64f6189066" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } health_report { overall_health: UNKNOWN } } }
Aug 11 15:08:25 dist-test-slave-1vmg krb5kdc[1132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460903, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:25.021531 2072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:25.012913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45639 (local address 127.0.0.1:52720)
0811 15:08:25.013479 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:25.013494 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:25.013612 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:25.014026 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:25.014029 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:25.014645 (+ 616us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:25.014764 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:25.014771 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.015567 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.015569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:25.016138 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:25.016146 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.016512 (+ 366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.016515 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:25.016575 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:25.017140 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:25.017152 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:25.018633 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:25.020138 (+ 1505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:25.020148 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:25.020154 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:25.020388 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:25.020605 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:25.020606 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:25.020608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:25.020678 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:25.020989 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:25.020995 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:25.021110 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 15:08:25.021200 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 15:08:25.021336 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":130,"threads_started":1}
I20260811 15:08:25.021530 1274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:25.013242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52720 (local address 127.30.8.62:45639)
0811 15:08:25.013472 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:25.013476 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:25.013565 (+ 89us) server_negotiation.cc:407] Connection header received
0811 15:08:25.013746 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:25.013749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:25.013793 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:25.013869 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:25.014907 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.015480 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.016232 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.016431 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.016499 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:25.018779 (+ 2280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:25.018782 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:25.018784 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:25.018808 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:25.020036 (+ 1228us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:25.020465 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:25.020468 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:25.020469 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:25.020534 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:25.020737 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:25.020739 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:25.020740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:25.020915 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:25.020988 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:25.021256 (+ 268us) server_negotiation.cc:299] Negotiation successful
0811 15:08:25.021395 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":147}
I20260811 15:08:25.025703 1193 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:52720:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 15:08:25.026472 1193 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:08:25.031700 1425 tablet_service.cc:1467] Processing CreateTablet for tablet 0303ecc3a03741b69795393d2f8c6535 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d4193f46cb7946b899c563f0581eb553]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 15:08:25.031848 1897 tablet_service.cc:1467] Processing CreateTablet for tablet 0303ecc3a03741b69795393d2f8c6535 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d4193f46cb7946b899c563f0581eb553]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 15:08:25.031800 1664 tablet_service.cc:1467] Processing CreateTablet for tablet 0303ecc3a03741b69795393d2f8c6535 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d4193f46cb7946b899c563f0581eb553]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 15:08:25.032511 1425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0303ecc3a03741b69795393d2f8c6535. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:25.032532 1897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0303ecc3a03741b69795393d2f8c6535. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:25.032532 1664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0303ecc3a03741b69795393d2f8c6535. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:25.035215 2044 tablet_bootstrap.cc:492] T 0303ecc3a03741b69795393d2f8c6535 P 047b159c89f543b185fc7da705f3eac5: Bootstrap starting.
I20260811 15:08:25.035501 2049 tablet_bootstrap.cc:492] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3: Bootstrap starting.
I20260811 15:08:25.035727 2048 tablet_bootstrap.cc:492] T 0303ecc3a03741b69795393d2f8c6535 P a5680e80a3cf4cfa8e872f64f6189066: Bootstrap starting.
I20260811 15:08:25.036424 2049 tablet_bootstrap.cc:654] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:25.036427 2044 tablet_bootstrap.cc:654] T 0303ecc3a03741b69795393d2f8c6535 P 047b159c89f543b185fc7da705f3eac5: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:25.036718 2048 tablet_bootstrap.cc:654] T 0303ecc3a03741b69795393d2f8c6535 P a5680e80a3cf4cfa8e872f64f6189066: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:25.037905 2044 tablet_bootstrap.cc:492] T 0303ecc3a03741b69795393d2f8c6535 P 047b159c89f543b185fc7da705f3eac5: No bootstrap required, opened a new log
I20260811 15:08:25.037997 2044 ts_tablet_manager.cc:1397] T 0303ecc3a03741b69795393d2f8c6535 P 047b159c89f543b185fc7da705f3eac5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 15:08:25.038218 2049 tablet_bootstrap.cc:492] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3: No bootstrap required, opened a new log
I20260811 15:08:25.038312 2049 ts_tablet_manager.cc:1397] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 15:08:25.038589 2048 tablet_bootstrap.cc:492] T 0303ecc3a03741b69795393d2f8c6535 P a5680e80a3cf4cfa8e872f64f6189066: No bootstrap required, opened a new log
I20260811 15:08:25.038493 2044 raft_consensus.cc:348] T 0303ecc3a03741b69795393d2f8c6535 P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:25.038640 2044 raft_consensus.cc:374] T 0303ecc3a03741b69795393d2f8c6535 P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:25.038667 2048 ts_tablet_manager.cc:1397] T 0303ecc3a03741b69795393d2f8c6535 P a5680e80a3cf4cfa8e872f64f6189066: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 15:08:25.038676 2044 raft_consensus.cc:729] T 0303ecc3a03741b69795393d2f8c6535 P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 047b159c89f543b185fc7da705f3eac5, State: Initialized, Role: FOLLOWER
I20260811 15:08:25.038735 2049 raft_consensus.cc:348] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:25.038841 2049 raft_consensus.cc:374] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:25.038813 2044 consensus_queue.cc:260] T 0303ecc3a03741b69795393d2f8c6535 P 047b159c89f543b185fc7da705f3eac5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:25.038870 2049 raft_consensus.cc:729] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6d0487c1f6145f0aa0e09ed071cb3c3, State: Initialized, Role: FOLLOWER
I20260811 15:08:25.039031 2049 consensus_queue.cc:260] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:25.039104 2044 ts_tablet_manager.cc:1428] T 0303ecc3a03741b69795393d2f8c6535 P 047b159c89f543b185fc7da705f3eac5: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:25.039122 2056 tablet_replica.cc:439] T 0303ecc3a03741b69795393d2f8c6535 P 047b159c89f543b185fc7da705f3eac5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } } }
I20260811 15:08:25.039167 2048 raft_consensus.cc:348] T 0303ecc3a03741b69795393d2f8c6535 P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:25.039265 2048 raft_consensus.cc:374] T 0303ecc3a03741b69795393d2f8c6535 P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:25.039263 2056 tablet_replica.cc:442] T 0303ecc3a03741b69795393d2f8c6535 P 047b159c89f543b185fc7da705f3eac5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:25.039292 2048 raft_consensus.cc:729] T 0303ecc3a03741b69795393d2f8c6535 P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5680e80a3cf4cfa8e872f64f6189066, State: Initialized, Role: FOLLOWER
I20260811 15:08:25.039314 2049 ts_tablet_manager.cc:1428] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:25.039399 2048 consensus_queue.cc:260] T 0303ecc3a03741b69795393d2f8c6535 P a5680e80a3cf4cfa8e872f64f6189066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:25.039645 2048 ts_tablet_manager.cc:1428] T 0303ecc3a03741b69795393d2f8c6535 P a5680e80a3cf4cfa8e872f64f6189066: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:25.039664 2057 tablet_replica.cc:439] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } } }
I20260811 15:08:25.039832 2057 tablet_replica.cc:442] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:25.039939 2058 tablet_replica.cc:439] T 0303ecc3a03741b69795393d2f8c6535 P a5680e80a3cf4cfa8e872f64f6189066: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } } }
I20260811 15:08:25.040045 2058 tablet_replica.cc:442] T 0303ecc3a03741b69795393d2f8c6535 P a5680e80a3cf4cfa8e872f64f6189066: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:25.353578 2058 consensus_queue.cc:1035] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [LEADER]: Connected to new peer: Peer: permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:25.359977 2079 consensus_queue.cc:1035] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:25.493112 2057 raft_consensus.cc:482] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:08:25.493268 2057 raft_consensus.cc:504] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:25.494508 2057 leader_election.cc:288] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 047b159c89f543b185fc7da705f3eac5 (127.30.8.1:33327), a5680e80a3cf4cfa8e872f64f6189066 (127.30.8.2:44741)
I20260811 15:08:25.498499 2086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:25.494855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:35311 (local address 127.30.8.1:33327)
0811 15:08:25.495167 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:25.495172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:25.495196 (+ 24us) server_negotiation.cc:407] Connection header received
0811 15:08:25.495378 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:25.495382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:25.495428 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:25.495491 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:25.495846 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.496361 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.497514 (+ 1153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.497952 (+ 438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.498052 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:25.498118 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:25.498299 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 15:08:25.498376 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":120,"threads_started":1}
I20260811 15:08:25.498499 2090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:25.494982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:39495 (local address 127.30.8.2:44741)
0811 15:08:25.495211 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:25.495215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:25.495292 (+ 77us) server_negotiation.cc:407] Connection header received
0811 15:08:25.495402 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:25.495404 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:25.495445 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:25.495526 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:25.495852 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.496361 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.497523 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.497929 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.498013 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:25.498084 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:25.498279 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 15:08:25.498351 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":70,"threads_started":1}
I20260811 15:08:25.498502 2085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:25.494741 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:33327 (local address 127.30.8.3:35311)
0811 15:08:25.495129 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:25.495144 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:25.495258 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:25.495526 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:25.495529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:25.495545 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:25.495734 (+ 189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:25.495741 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.496474 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.496477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:25.497388 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:25.497398 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.498083 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.498086 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:25.498182 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:25.498200 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 15:08:25.498271 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 15:08:25.498348 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":305,"mutex_wait_us":62,"spinlock_wait_cycles":3840,"thread_start_us":86,"threads_started":1}
I20260811 15:08:25.498512 2084 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:25.494785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:44741 (local address 127.30.8.3:39495)
0811 15:08:25.495179 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:25.495192 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:25.495303 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:25.495545 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:25.495548 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:25.495592 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:25.495736 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:25.495741 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.496471 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.496474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:25.497392 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:25.497402 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.498048 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.498051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:25.498146 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:25.498170 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 15:08:25.498239 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 15:08:25.498324 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":70,"spinlock_wait_cycles":36352,"thread_start_us":104,"threads_started":1}
I20260811 15:08:25.499042 1460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0303ecc3a03741b69795393d2f8c6535" candidate_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047b159c89f543b185fc7da705f3eac5" is_pre_election: true
I20260811 15:08:25.499207 1460 raft_consensus.cc:2454] T 0303ecc3a03741b69795393d2f8c6535 P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6d0487c1f6145f0aa0e09ed071cb3c3 in term 0.
I20260811 15:08:25.499573 1692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0303ecc3a03741b69795393d2f8c6535" candidate_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5680e80a3cf4cfa8e872f64f6189066" is_pre_election: true
I20260811 15:08:25.499593 1820 leader_election.cc:302] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047b159c89f543b185fc7da705f3eac5, b6d0487c1f6145f0aa0e09ed071cb3c3; no voters:
I20260811 15:08:25.499806 1692 raft_consensus.cc:2454] T 0303ecc3a03741b69795393d2f8c6535 P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6d0487c1f6145f0aa0e09ed071cb3c3 in term 0.
I20260811 15:08:25.499881 2057 raft_consensus.cc:2781] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:08:25.499944 2057 raft_consensus.cc:482] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:08:25.499986 2057 raft_consensus.cc:3037] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:25.501089 2057 raft_consensus.cc:504] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:25.501456 2057 leader_election.cc:288] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3 [CANDIDATE]: Term 1 election: Requested vote from peers 047b159c89f543b185fc7da705f3eac5 (127.30.8.1:33327), a5680e80a3cf4cfa8e872f64f6189066 (127.30.8.2:44741)
I20260811 15:08:25.501796 1460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0303ecc3a03741b69795393d2f8c6535" candidate_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047b159c89f543b185fc7da705f3eac5"
I20260811 15:08:25.501909 1460 raft_consensus.cc:3037] T 0303ecc3a03741b69795393d2f8c6535 P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:25.502781 1460 raft_consensus.cc:2454] T 0303ecc3a03741b69795393d2f8c6535 P 047b159c89f543b185fc7da705f3eac5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6d0487c1f6145f0aa0e09ed071cb3c3 in term 1.
I20260811 15:08:25.503036 1692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0303ecc3a03741b69795393d2f8c6535" candidate_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5680e80a3cf4cfa8e872f64f6189066"
I20260811 15:08:25.503082 1820 leader_election.cc:302] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047b159c89f543b185fc7da705f3eac5, b6d0487c1f6145f0aa0e09ed071cb3c3; no voters:
I20260811 15:08:25.503147 1692 raft_consensus.cc:3037] T 0303ecc3a03741b69795393d2f8c6535 P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:25.503257 2057 raft_consensus.cc:2781] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:25.503437 2057 raft_consensus.cc:686] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 LEADER]: Becoming Leader. State: Replica: b6d0487c1f6145f0aa0e09ed071cb3c3, State: Running, Role: LEADER
I20260811 15:08:25.503798 2057 consensus_queue.cc:237] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:25.504134 1692 raft_consensus.cc:2454] T 0303ecc3a03741b69795393d2f8c6535 P a5680e80a3cf4cfa8e872f64f6189066 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6d0487c1f6145f0aa0e09ed071cb3c3 in term 1.
I20260811 15:08:25.505072 2091 tablet_replica.cc:439] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3: TxnStatusTablet state changed. Reason: New leader b6d0487c1f6145f0aa0e09ed071cb3c3. Latest consensus state: current_term: 1 leader_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } } }
I20260811 15:08:25.505187 2091 tablet_replica.cc:442] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3: This TxnStatusTablet replica's current role is: LEADER
I20260811 15:08:25.505654 2094 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 15:08:25.505834 2094 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 15:08:25.506287 1194 catalog_manager.cc:5495] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6d0487c1f6145f0aa0e09ed071cb3c3 (127.30.8.3). New cstate: current_term: 1 leader_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } health_report { overall_health: HEALTHY } } }
I20260811 15:08:25.611953 2100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:25.608024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:33441 (local address 127.0.0.1:49264)
0811 15:08:25.608330 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:25.608348 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:25.608473 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:25.608830 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:25.608832 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:25.608840 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 15:08:25.608946 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:25.608951 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.609679 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.609683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:25.610330 (+ 647us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:25.610339 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.610782 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.610786 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:25.610871 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:25.610960 (+ 89us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 15:08:25.611636 (+ 676us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 15:08:25.611653 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 15:08:25.611718 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 15:08:25.611814 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":94,"threads_started":1}
I20260811 15:08:25.611953 2102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:25.608110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49264 (local address 127.30.8.3:33441)
0811 15:08:25.608391 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:25.608396 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:25.608411 (+ 15us) server_negotiation.cc:407] Connection header received
0811 15:08:25.608621 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:25.608623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:25.608689 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:25.608792 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:25.609045 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.609567 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.610446 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.610673 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.610763 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:25.611114 (+ 351us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:25.611532 (+ 418us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:25.611637 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:25.611729 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 15:08:25.611789 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":101,"threads_started":1}
I20260811 15:08:25.617580 1460 raft_consensus.cc:1261] T 0303ecc3a03741b69795393d2f8c6535 P 047b159c89f543b185fc7da705f3eac5 [term 1 FOLLOWER]: Refusing update from remote peer b6d0487c1f6145f0aa0e09ed071cb3c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:25.617748 1692 raft_consensus.cc:1261] T 0303ecc3a03741b69795393d2f8c6535 P a5680e80a3cf4cfa8e872f64f6189066 [term 1 FOLLOWER]: Refusing update from remote peer b6d0487c1f6145f0aa0e09ed071cb3c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:25.618180 2057 consensus_queue.cc:1035] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:25.618391 2091 consensus_queue.cc:1035] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:25.621819 2056 tablet_replica.cc:439] T 0303ecc3a03741b69795393d2f8c6535 P 047b159c89f543b185fc7da705f3eac5: TxnStatusTablet state changed. Reason: New leader b6d0487c1f6145f0aa0e09ed071cb3c3. Latest consensus state: current_term: 1 leader_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } } }
I20260811 15:08:25.621960 2056 tablet_replica.cc:442] T 0303ecc3a03741b69795393d2f8c6535 P 047b159c89f543b185fc7da705f3eac5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:25.623239 2079 tablet_replica.cc:439] T 0303ecc3a03741b69795393d2f8c6535 P a5680e80a3cf4cfa8e872f64f6189066: TxnStatusTablet state changed. Reason: New leader b6d0487c1f6145f0aa0e09ed071cb3c3. Latest consensus state: current_term: 1 leader_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } } }
I20260811 15:08:25.623376 2079 tablet_replica.cc:442] T 0303ecc3a03741b69795393d2f8c6535 P a5680e80a3cf4cfa8e872f64f6189066: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:25.624722 2057 tablet_replica.cc:439] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } } }
I20260811 15:08:25.624847 2057 tablet_replica.cc:442] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3: This TxnStatusTablet replica's current role is: LEADER
I20260811 15:08:25.624951 2057 tablet_replica.cc:439] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } } }
I20260811 15:08:25.625016 2057 tablet_replica.cc:442] T 0303ecc3a03741b69795393d2f8c6535 P b6d0487c1f6145f0aa0e09ed071cb3c3: This TxnStatusTablet replica's current role is: LEADER
I20260811 15:08:25.625094 2083 mvcc.cc:205] Tried to move back new op lower bound from 7317343869403377664 to 7317343868946071552. Current Snapshot: MvccSnapshot[applied={T|T < 7317343869403377664}]
I20260811 15:08:25.625964 2081 mvcc.cc:205] Tried to move back new op lower bound from 7317343869403377664 to 7317343868946071552. Current Snapshot: MvccSnapshot[applied={T|T < 7317343869403377664}]
I20260811 15:08:25.626142 2056 tablet_replica.cc:439] T 0303ecc3a03741b69795393d2f8c6535 P 047b159c89f543b185fc7da705f3eac5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } } }
I20260811 15:08:25.626243 2056 tablet_replica.cc:442] T 0303ecc3a03741b69795393d2f8c6535 P 047b159c89f543b185fc7da705f3eac5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:25.627846 2079 tablet_replica.cc:439] T 0303ecc3a03741b69795393d2f8c6535 P a5680e80a3cf4cfa8e872f64f6189066: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } } }
I20260811 15:08:25.627966 2079 tablet_replica.cc:442] T 0303ecc3a03741b69795393d2f8c6535 P a5680e80a3cf4cfa8e872f64f6189066: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:25.635820 2090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:25.632145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55812 (local address 127.30.8.2:44741)
0811 15:08:25.632275 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:25.632280 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:25.632466 (+ 186us) server_negotiation.cc:407] Connection header received
0811 15:08:25.632611 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:25.632614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:25.632673 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:25.632760 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:25.632996 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.633504 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.634362 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.634582 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.634674 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:25.634956 (+ 282us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:25.635413 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:25.635523 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:25.635615 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 15:08:25.635673 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 15:08:25.649263 2102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:25.644741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:60635 (local address 127.30.8.3:33441)
0811 15:08:25.644871 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:25.644877 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:25.645119 (+ 242us) server_negotiation.cc:407] Connection header received
0811 15:08:25.645247 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:25.645251 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:25.645309 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:25.645402 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:25.645666 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.646465 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.647886 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.648270 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.648383 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:25.648793 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:25.648888 (+ 95us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:25.648961 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:25.649044 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 15:08:25.649107 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 15:08:25.649322 2123 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:25.644625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:33441 (local address 127.30.8.2:60635)
0811 15:08:25.645022 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:25.645035 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:25.645144 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:25.645419 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:25.645421 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:25.645430 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 15:08:25.645549 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:25.645555 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.646598 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.646601 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:25.647752 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:25.647763 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.648389 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.648392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:25.648521 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:25.648593 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 15:08:25.648975 (+ 382us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 15:08:25.648980 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 15:08:25.649033 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 15:08:25.649117 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":105,"threads_started":1}
I20260811 15:08:25.678465 2129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:25.674485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:44741 (local address 127.30.8.3:51225)
0811 15:08:25.674846 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:25.674862 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:25.675000 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:25.675241 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:25.675244 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:25.675251 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 15:08:25.675359 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:25.675365 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.676290 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.676293 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:25.677145 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:25.677156 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.677726 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.677730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:25.677813 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:25.677870 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 15:08:25.678183 (+ 313us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 15:08:25.678191 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 15:08:25.678259 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 15:08:25.678339 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":138,"threads_started":1}
I20260811 15:08:25.678465 2090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:25.674816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:51225 (local address 127.30.8.2:44741)
0811 15:08:25.674987 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:25.674991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:25.675006 (+ 15us) server_negotiation.cc:407] Connection header received
0811 15:08:25.675091 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:25.675094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:25.675141 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:25.675202 (+ 61us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:25.675455 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.676162 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.677252 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.677616 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.677736 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:25.677993 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:25.678105 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:25.678177 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:25.678271 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 15:08:25.678328 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 15:08:25.702581 1195 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:25.702780 1195 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:25.705328 1195 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365: Sending DeleteTablet for 3 replicas of tablet 992ef160a1cf445a905670b00de55fc7
I20260811 15:08:25.706259 1664 tablet_service.cc:1514] Processing DeleteTablet for tablet 992ef160a1cf445a905670b00de55fc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:25 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:55802
I20260811 15:08:25.706259 1425 tablet_service.cc:1514] Processing DeleteTablet for tablet 992ef160a1cf445a905670b00de55fc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:25 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:43890
I20260811 15:08:25.706554 1897 tablet_service.cc:1514] Processing DeleteTablet for tablet 992ef160a1cf445a905670b00de55fc7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:25 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:49256
I20260811 15:08:25.706857 2136 tablet_replica.cc:330] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066: stopping tablet replica
I20260811 15:08:25.707125 2139 tablet_replica.cc:330] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3: stopping tablet replica
I20260811 15:08:25.707249 2136 raft_consensus.cc:2229] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:08:25.707429 2139 raft_consensus.cc:2229] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:25.707621 2139 raft_consensus.cc:2258] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:25.707666 2136 raft_consensus.cc:2258] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:25.707765 1195 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52704:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:08:25.708246 1195 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:08:25.709035 2139 ts_tablet_manager.cc:1905] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:25.709040 2136 ts_tablet_manager.cc:1905] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:25.712323 2136 ts_tablet_manager.cc:1918] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 15:08:25.712365 2139 ts_tablet_manager.cc:1918] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 15:08:25.712435 2136 log.cc:1196] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/992ef160a1cf445a905670b00de55fc7
I20260811 15:08:25.712457 2139 log.cc:1196] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/992ef160a1cf445a905670b00de55fc7
I20260811 15:08:25.712782 2136 ts_tablet_manager.cc:1939] T 992ef160a1cf445a905670b00de55fc7 P a5680e80a3cf4cfa8e872f64f6189066: Deleting consensus metadata
I20260811 15:08:25.712806 2139 ts_tablet_manager.cc:1939] T 992ef160a1cf445a905670b00de55fc7 P b6d0487c1f6145f0aa0e09ed071cb3c3: Deleting consensus metadata
I20260811 15:08:25.714082 1176 catalog_manager.cc:4841] TS a5680e80a3cf4cfa8e872f64f6189066 (127.30.8.2:44741): tablet 992ef160a1cf445a905670b00de55fc7 (table test-table [id=cf88fe8390634e0b9f80707f2509effe]) successfully deleted
I20260811 15:08:25.714258 1176 catalog_manager.cc:4841] TS b6d0487c1f6145f0aa0e09ed071cb3c3 (127.30.8.3:33441): tablet 992ef160a1cf445a905670b00de55fc7 (table test-table [id=cf88fe8390634e0b9f80707f2509effe]) successfully deleted
I20260811 15:08:25.714507 2138 tablet_replica.cc:330] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5: stopping tablet replica
I20260811 15:08:25.714860 2138 raft_consensus.cc:2229] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:25.714908 1897 tablet_service.cc:1467] Processing CreateTablet for tablet b776992a4aef454c87a2849600560c0c (DEFAULT_TABLE table=test-table [id=24e5605657c14d6db07fa8f5447ac5b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:25.715111 2138 raft_consensus.cc:2258] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:25.715220 1897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b776992a4aef454c87a2849600560c0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:25.715332 1425 tablet_service.cc:1467] Processing CreateTablet for tablet b776992a4aef454c87a2849600560c0c (DEFAULT_TABLE table=test-table [id=24e5605657c14d6db07fa8f5447ac5b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:25.715579 1425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b776992a4aef454c87a2849600560c0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:25.717093 2138 ts_tablet_manager.cc:1905] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:25.717993 1664 tablet_service.cc:1467] Processing CreateTablet for tablet b776992a4aef454c87a2849600560c0c (DEFAULT_TABLE table=test-table [id=24e5605657c14d6db07fa8f5447ac5b6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:25.718268 1664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b776992a4aef454c87a2849600560c0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:25.719012 2141 tablet_bootstrap.cc:492] T b776992a4aef454c87a2849600560c0c P 047b159c89f543b185fc7da705f3eac5: Bootstrap starting.
I20260811 15:08:25.719995 2138 ts_tablet_manager.cc:1918] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 15:08:25.720094 2138 log.cc:1196] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-0/wal/wals/992ef160a1cf445a905670b00de55fc7
I20260811 15:08:25.720412 2138 ts_tablet_manager.cc:1939] T 992ef160a1cf445a905670b00de55fc7 P 047b159c89f543b185fc7da705f3eac5: Deleting consensus metadata
I20260811 15:08:25.721498 2141 tablet_bootstrap.cc:654] T b776992a4aef454c87a2849600560c0c P 047b159c89f543b185fc7da705f3eac5: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:25.721535 2140 tablet_bootstrap.cc:492] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3: Bootstrap starting.
I20260811 15:08:25.721756 1177 catalog_manager.cc:4841] TS 047b159c89f543b185fc7da705f3eac5 (127.30.8.1:33327): tablet 992ef160a1cf445a905670b00de55fc7 (table test-table [id=cf88fe8390634e0b9f80707f2509effe]) successfully deleted
I20260811 15:08:25.722453 2140 tablet_bootstrap.cc:654] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:25.722944 2142 tablet_bootstrap.cc:492] T b776992a4aef454c87a2849600560c0c P a5680e80a3cf4cfa8e872f64f6189066: Bootstrap starting.
I20260811 15:08:25.723589 2140 tablet_bootstrap.cc:492] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3: No bootstrap required, opened a new log
I20260811 15:08:25.723686 2140 ts_tablet_manager.cc:1397] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 15:08:25.723919 2142 tablet_bootstrap.cc:654] T b776992a4aef454c87a2849600560c0c P a5680e80a3cf4cfa8e872f64f6189066: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:25.724154 2140 raft_consensus.cc:348] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:25.724294 2140 raft_consensus.cc:374] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:25.724335 2140 raft_consensus.cc:729] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6d0487c1f6145f0aa0e09ed071cb3c3, State: Initialized, Role: FOLLOWER
I20260811 15:08:25.724490 2141 tablet_bootstrap.cc:492] T b776992a4aef454c87a2849600560c0c P 047b159c89f543b185fc7da705f3eac5: No bootstrap required, opened a new log
I20260811 15:08:25.724466 2140 consensus_queue.cc:260] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:25.724687 2141 ts_tablet_manager.cc:1397] T b776992a4aef454c87a2849600560c0c P 047b159c89f543b185fc7da705f3eac5: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.001s
I20260811 15:08:25.724768 2140 ts_tablet_manager.cc:1428] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:25.725210 2142 tablet_bootstrap.cc:492] T b776992a4aef454c87a2849600560c0c P a5680e80a3cf4cfa8e872f64f6189066: No bootstrap required, opened a new log
I20260811 15:08:25.725302 2142 ts_tablet_manager.cc:1397] T b776992a4aef454c87a2849600560c0c P a5680e80a3cf4cfa8e872f64f6189066: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260811 15:08:25.725194 2141 raft_consensus.cc:348] T b776992a4aef454c87a2849600560c0c P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:25.725472 2141 raft_consensus.cc:374] T b776992a4aef454c87a2849600560c0c P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:25.725555 2141 raft_consensus.cc:729] T b776992a4aef454c87a2849600560c0c P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 047b159c89f543b185fc7da705f3eac5, State: Initialized, Role: FOLLOWER
I20260811 15:08:25.725742 2142 raft_consensus.cc:348] T b776992a4aef454c87a2849600560c0c P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:25.725862 2142 raft_consensus.cc:374] T b776992a4aef454c87a2849600560c0c P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:25.725896 2142 raft_consensus.cc:729] T b776992a4aef454c87a2849600560c0c P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5680e80a3cf4cfa8e872f64f6189066, State: Initialized, Role: FOLLOWER
I20260811 15:08:25.726022 2142 consensus_queue.cc:260] T b776992a4aef454c87a2849600560c0c P a5680e80a3cf4cfa8e872f64f6189066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:25.726292 2142 ts_tablet_manager.cc:1428] T b776992a4aef454c87a2849600560c0c P a5680e80a3cf4cfa8e872f64f6189066: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 15:08:25.725765 2141 consensus_queue.cc:260] T b776992a4aef454c87a2849600560c0c P 047b159c89f543b185fc7da705f3eac5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:25.728096 2141 ts_tablet_manager.cc:1428] T b776992a4aef454c87a2849600560c0c P 047b159c89f543b185fc7da705f3eac5: Time spent starting tablet: real 0.003s user 0.000s sys 0.001s
I20260811 15:08:25.927745 2132 raft_consensus.cc:482] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:08:25.927912 2132 raft_consensus.cc:504] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:25.928406 2132 leader_election.cc:288] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 047b159c89f543b185fc7da705f3eac5 (127.30.8.1:33327), a5680e80a3cf4cfa8e872f64f6189066 (127.30.8.2:44741)
I20260811 15:08:25.928903 1460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b776992a4aef454c87a2849600560c0c" candidate_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047b159c89f543b185fc7da705f3eac5" is_pre_election: true
I20260811 15:08:25.928902 1692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b776992a4aef454c87a2849600560c0c" candidate_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5680e80a3cf4cfa8e872f64f6189066" is_pre_election: true
I20260811 15:08:25.929049 1460 raft_consensus.cc:2454] T b776992a4aef454c87a2849600560c0c P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6d0487c1f6145f0aa0e09ed071cb3c3 in term 0.
I20260811 15:08:25.929072 1692 raft_consensus.cc:2454] T b776992a4aef454c87a2849600560c0c P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6d0487c1f6145f0aa0e09ed071cb3c3 in term 0.
I20260811 15:08:25.929385 1820 leader_election.cc:302] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047b159c89f543b185fc7da705f3eac5, b6d0487c1f6145f0aa0e09ed071cb3c3; no voters:
I20260811 15:08:25.929595 2132 raft_consensus.cc:2781] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:08:25.929657 2132 raft_consensus.cc:482] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:08:25.929684 2132 raft_consensus.cc:3037] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:25.930701 2132 raft_consensus.cc:504] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:25.931084 2132 leader_election.cc:288] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [CANDIDATE]: Term 1 election: Requested vote from peers 047b159c89f543b185fc7da705f3eac5 (127.30.8.1:33327), a5680e80a3cf4cfa8e872f64f6189066 (127.30.8.2:44741)
I20260811 15:08:25.931496 1460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b776992a4aef454c87a2849600560c0c" candidate_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047b159c89f543b185fc7da705f3eac5"
I20260811 15:08:25.931500 1692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b776992a4aef454c87a2849600560c0c" candidate_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5680e80a3cf4cfa8e872f64f6189066"
I20260811 15:08:25.931627 1460 raft_consensus.cc:3037] T b776992a4aef454c87a2849600560c0c P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:25.931632 1692 raft_consensus.cc:3037] T b776992a4aef454c87a2849600560c0c P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:25.932578 1460 raft_consensus.cc:2454] T b776992a4aef454c87a2849600560c0c P 047b159c89f543b185fc7da705f3eac5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6d0487c1f6145f0aa0e09ed071cb3c3 in term 1.
I20260811 15:08:25.932600 1692 raft_consensus.cc:2454] T b776992a4aef454c87a2849600560c0c P a5680e80a3cf4cfa8e872f64f6189066 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6d0487c1f6145f0aa0e09ed071cb3c3 in term 1.
I20260811 15:08:25.932871 1820 leader_election.cc:302] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047b159c89f543b185fc7da705f3eac5, b6d0487c1f6145f0aa0e09ed071cb3c3; no voters:
I20260811 15:08:25.933051 2132 raft_consensus.cc:2781] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:25.933142 2132 raft_consensus.cc:686] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 LEADER]: Becoming Leader. State: Replica: b6d0487c1f6145f0aa0e09ed071cb3c3, State: Running, Role: LEADER
I20260811 15:08:25.933296 2132 consensus_queue.cc:237] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:25.935128 1195 catalog_manager.cc:5495] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6d0487c1f6145f0aa0e09ed071cb3c3 (127.30.8.3). New cstate: current_term: 1 leader_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } health_report { overall_health: HEALTHY } } }
I20260811 15:08:25.973181 2102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:25.970175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49270 (local address 127.30.8.3:33441)
0811 15:08:25.970286 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:25.970290 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:25.970303 (+ 13us) server_negotiation.cc:407] Connection header received
0811 15:08:25.970460 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:25.970463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:25.970513 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:25.970580 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:25.970813 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.971351 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.972172 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.972398 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.972487 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:25.972733 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:25.972814 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:25.972873 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:25.972973 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 15:08:25.973028 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260811 15:08:25.983815 2102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:25.980790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:51115 (local address 127.30.8.3:33441)
0811 15:08:25.980897 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:25.980902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:25.980959 (+ 57us) server_negotiation.cc:407] Connection header received
0811 15:08:25.981121 (+ 162us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:25.981124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:25.981164 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:25.981226 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:25.981467 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.981974 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.982755 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.983007 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.983100 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:25.983338 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:25.983413 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:25.983472 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:25.983565 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 15:08:25.983638 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":52}
I20260811 15:08:25.983815 2129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:25.980705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:33441 (local address 127.30.8.3:51115)
0811 15:08:25.980889 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:25.980908 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:25.981020 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:25.981244 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:25.981247 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:25.981254 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 15:08:25.981373 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:25.981379 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.982086 (+ 707us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.982089 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:25.982650 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:25.982657 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:25.983122 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:25.983125 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:25.983207 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:25.983233 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 15:08:25.983489 (+ 256us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 15:08:25.983495 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 15:08:25.983555 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 15:08:25.983656 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":64,"mutex_wait_us":20}
I20260811 15:08:25.990729 1460 raft_consensus.cc:1261] T b776992a4aef454c87a2849600560c0c P 047b159c89f543b185fc7da705f3eac5 [term 1 FOLLOWER]: Refusing update from remote peer b6d0487c1f6145f0aa0e09ed071cb3c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:25.990718 1692 raft_consensus.cc:1261] T b776992a4aef454c87a2849600560c0c P a5680e80a3cf4cfa8e872f64f6189066 [term 1 FOLLOWER]: Refusing update from remote peer b6d0487c1f6145f0aa0e09ed071cb3c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:25.991153 2132 consensus_queue.cc:1035] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:25.991302 2057 consensus_queue.cc:1035] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 15:08:26.006702 2083 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 15:08:27.568987 1195 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:27.569164 1195 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52704:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:27.571717 1195 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365: Sending DeleteTablet for 3 replicas of tablet b776992a4aef454c87a2849600560c0c
I20260811 15:08:27.572634 1664 tablet_service.cc:1514] Processing DeleteTablet for tablet b776992a4aef454c87a2849600560c0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:27 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:55802
I20260811 15:08:27.572672 1425 tablet_service.cc:1514] Processing DeleteTablet for tablet b776992a4aef454c87a2849600560c0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:27 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:43890
I20260811 15:08:27.572993 1897 tablet_service.cc:1514] Processing DeleteTablet for tablet b776992a4aef454c87a2849600560c0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:27 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:49256
I20260811 15:08:27.573046 2190 tablet_replica.cc:330] T b776992a4aef454c87a2849600560c0c P 047b159c89f543b185fc7da705f3eac5: stopping tablet replica
I20260811 15:08:27.573179 2190 raft_consensus.cc:2229] T b776992a4aef454c87a2849600560c0c P 047b159c89f543b185fc7da705f3eac5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:27.573287 2190 raft_consensus.cc:2258] T b776992a4aef454c87a2849600560c0c P 047b159c89f543b185fc7da705f3eac5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:27.575127 2190 ts_tablet_manager.cc:1905] T b776992a4aef454c87a2849600560c0c P 047b159c89f543b185fc7da705f3eac5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:27.575585 2192 tablet_replica.cc:330] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3: stopping tablet replica
I20260811 15:08:27.575737 2192 raft_consensus.cc:2229] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:08:27.576121 2192 raft_consensus.cc:2258] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:27.576412 2189 tablet_replica.cc:330] T b776992a4aef454c87a2849600560c0c P a5680e80a3cf4cfa8e872f64f6189066: stopping tablet replica
I20260811 15:08:27.576610 2189 raft_consensus.cc:2229] T b776992a4aef454c87a2849600560c0c P a5680e80a3cf4cfa8e872f64f6189066 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:27.576752 2189 raft_consensus.cc:2258] T b776992a4aef454c87a2849600560c0c P a5680e80a3cf4cfa8e872f64f6189066 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:27.577504 2192 ts_tablet_manager.cc:1905] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:27.578104 2189 ts_tablet_manager.cc:1905] T b776992a4aef454c87a2849600560c0c P a5680e80a3cf4cfa8e872f64f6189066: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:27.578261 2190 ts_tablet_manager.cc:1918] T b776992a4aef454c87a2849600560c0c P 047b159c89f543b185fc7da705f3eac5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 15:08:27.578343 2190 log.cc:1196] T b776992a4aef454c87a2849600560c0c P 047b159c89f543b185fc7da705f3eac5: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-0/wal/wals/b776992a4aef454c87a2849600560c0c
I20260811 15:08:27.578693 2190 ts_tablet_manager.cc:1939] T b776992a4aef454c87a2849600560c0c P 047b159c89f543b185fc7da705f3eac5: Deleting consensus metadata
I20260811 15:08:27.579528 1177 catalog_manager.cc:4841] TS 047b159c89f543b185fc7da705f3eac5 (127.30.8.1:33327): tablet b776992a4aef454c87a2849600560c0c (table test-table [id=24e5605657c14d6db07fa8f5447ac5b6]) successfully deleted
I20260811 15:08:27.580861 2189 ts_tablet_manager.cc:1918] T b776992a4aef454c87a2849600560c0c P a5680e80a3cf4cfa8e872f64f6189066: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 15:08:27.580861 2192 ts_tablet_manager.cc:1918] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 15:08:27.580948 2189 log.cc:1196] T b776992a4aef454c87a2849600560c0c P a5680e80a3cf4cfa8e872f64f6189066: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/b776992a4aef454c87a2849600560c0c
I20260811 15:08:27.580948 2192 log.cc:1196] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/b776992a4aef454c87a2849600560c0c
I20260811 15:08:27.581248 2192 ts_tablet_manager.cc:1939] T b776992a4aef454c87a2849600560c0c P b6d0487c1f6145f0aa0e09ed071cb3c3: Deleting consensus metadata
I20260811 15:08:27.581229 2189 ts_tablet_manager.cc:1939] T b776992a4aef454c87a2849600560c0c P a5680e80a3cf4cfa8e872f64f6189066: Deleting consensus metadata
I20260811 15:08:27.582013 1176 catalog_manager.cc:4841] TS b6d0487c1f6145f0aa0e09ed071cb3c3 (127.30.8.3:33441): tablet b776992a4aef454c87a2849600560c0c (table test-table [id=24e5605657c14d6db07fa8f5447ac5b6]) successfully deleted
I20260811 15:08:27.582204 1176 catalog_manager.cc:4841] TS a5680e80a3cf4cfa8e872f64f6189066 (127.30.8.2:44741): tablet b776992a4aef454c87a2849600560c0c (table test-table [id=24e5605657c14d6db07fa8f5447ac5b6]) successfully deleted
Aug 11 15:08:27 dist-test-slave-1vmg krb5kdc[1132](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460907, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 15:08:27 dist-test-slave-1vmg krb5kdc[1132](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460907, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:27.604568 2215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:27.596336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52726 (local address 127.30.8.62:45639)
0811 15:08:27.596624 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:27.596631 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:27.596647 (+ 16us) server_negotiation.cc:407] Connection header received
0811 15:08:27.596699 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:27.596702 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:27.596747 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:27.596827 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:27.597527 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:27.598084 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:27.598852 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:27.599151 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:27.599229 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:27.601648 (+ 2419us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:27.601652 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:27.601654 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:27.601682 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:27.603180 (+ 1498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:27.603531 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:27.603534 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:27.603535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:27.603611 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:27.603797 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:27.603798 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:27.603800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:27.603989 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:27.604101 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:27.604278 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 15:08:27.604386 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":110,"threads_started":1}
I20260811 15:08:27.607273 1195 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:08:27.607756 1195 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:08:27.612541 1425 tablet_service.cc:1467] Processing CreateTablet for tablet 94d10407d26f4739bd155877c0d23a43 (DEFAULT_TABLE table=test-table [id=27d2122bc696488296cb6519cb0b96c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:27.612599 1897 tablet_service.cc:1467] Processing CreateTablet for tablet 94d10407d26f4739bd155877c0d23a43 (DEFAULT_TABLE table=test-table [id=27d2122bc696488296cb6519cb0b96c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:27.612573 1664 tablet_service.cc:1467] Processing CreateTablet for tablet 94d10407d26f4739bd155877c0d23a43 (DEFAULT_TABLE table=test-table [id=27d2122bc696488296cb6519cb0b96c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:27.612882 1897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94d10407d26f4739bd155877c0d23a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:27.612882 1664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94d10407d26f4739bd155877c0d23a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:27.612882 1425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 94d10407d26f4739bd155877c0d23a43. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:27.615005 2218 tablet_bootstrap.cc:492] T 94d10407d26f4739bd155877c0d23a43 P 047b159c89f543b185fc7da705f3eac5: Bootstrap starting.
I20260811 15:08:27.615586 2220 tablet_bootstrap.cc:492] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3: Bootstrap starting.
I20260811 15:08:27.616269 2218 tablet_bootstrap.cc:654] T 94d10407d26f4739bd155877c0d23a43 P 047b159c89f543b185fc7da705f3eac5: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:27.616468 2221 tablet_bootstrap.cc:492] T 94d10407d26f4739bd155877c0d23a43 P a5680e80a3cf4cfa8e872f64f6189066: Bootstrap starting.
I20260811 15:08:27.616516 2220 tablet_bootstrap.cc:654] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:27.617389 2221 tablet_bootstrap.cc:654] T 94d10407d26f4739bd155877c0d23a43 P a5680e80a3cf4cfa8e872f64f6189066: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:27.617705 2218 tablet_bootstrap.cc:492] T 94d10407d26f4739bd155877c0d23a43 P 047b159c89f543b185fc7da705f3eac5: No bootstrap required, opened a new log
I20260811 15:08:27.617713 2220 tablet_bootstrap.cc:492] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3: No bootstrap required, opened a new log
I20260811 15:08:27.617801 2220 ts_tablet_manager.cc:1397] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 15:08:27.617801 2218 ts_tablet_manager.cc:1397] T 94d10407d26f4739bd155877c0d23a43 P 047b159c89f543b185fc7da705f3eac5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 15:08:27.618243 2220 raft_consensus.cc:348] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:27.618243 2218 raft_consensus.cc:348] T 94d10407d26f4739bd155877c0d23a43 P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:27.618369 2218 raft_consensus.cc:374] T 94d10407d26f4739bd155877c0d23a43 P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:27.618405 2220 raft_consensus.cc:374] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:27.618410 2218 raft_consensus.cc:729] T 94d10407d26f4739bd155877c0d23a43 P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 047b159c89f543b185fc7da705f3eac5, State: Initialized, Role: FOLLOWER
I20260811 15:08:27.618440 2220 raft_consensus.cc:729] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6d0487c1f6145f0aa0e09ed071cb3c3, State: Initialized, Role: FOLLOWER
I20260811 15:08:27.618553 2218 consensus_queue.cc:260] T 94d10407d26f4739bd155877c0d23a43 P 047b159c89f543b185fc7da705f3eac5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:27.618582 2220 consensus_queue.cc:260] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:27.618647 2221 tablet_bootstrap.cc:492] T 94d10407d26f4739bd155877c0d23a43 P a5680e80a3cf4cfa8e872f64f6189066: No bootstrap required, opened a new log
I20260811 15:08:27.618732 2221 ts_tablet_manager.cc:1397] T 94d10407d26f4739bd155877c0d23a43 P a5680e80a3cf4cfa8e872f64f6189066: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 15:08:27.618848 2220 ts_tablet_manager.cc:1428] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:27.618943 2218 ts_tablet_manager.cc:1428] T 94d10407d26f4739bd155877c0d23a43 P 047b159c89f543b185fc7da705f3eac5: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 15:08:27.619174 2221 raft_consensus.cc:348] T 94d10407d26f4739bd155877c0d23a43 P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:27.619257 2221 raft_consensus.cc:374] T 94d10407d26f4739bd155877c0d23a43 P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:27.619280 2221 raft_consensus.cc:729] T 94d10407d26f4739bd155877c0d23a43 P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5680e80a3cf4cfa8e872f64f6189066, State: Initialized, Role: FOLLOWER
I20260811 15:08:27.619386 2221 consensus_queue.cc:260] T 94d10407d26f4739bd155877c0d23a43 P a5680e80a3cf4cfa8e872f64f6189066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:27.619707 2221 ts_tablet_manager.cc:1428] T 94d10407d26f4739bd155877c0d23a43 P a5680e80a3cf4cfa8e872f64f6189066: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:27.639983 2180 raft_consensus.cc:482] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:08:27.640161 2180 raft_consensus.cc:504] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:27.640630 2180 leader_election.cc:288] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 047b159c89f543b185fc7da705f3eac5 (127.30.8.1:33327), a5680e80a3cf4cfa8e872f64f6189066 (127.30.8.2:44741)
I20260811 15:08:27.641233 1692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94d10407d26f4739bd155877c0d23a43" candidate_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5680e80a3cf4cfa8e872f64f6189066" is_pre_election: true
I20260811 15:08:27.641216 1458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94d10407d26f4739bd155877c0d23a43" candidate_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047b159c89f543b185fc7da705f3eac5" is_pre_election: true
I20260811 15:08:27.641408 1692 raft_consensus.cc:2454] T 94d10407d26f4739bd155877c0d23a43 P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6d0487c1f6145f0aa0e09ed071cb3c3 in term 0.
I20260811 15:08:27.641497 1458 raft_consensus.cc:2454] T 94d10407d26f4739bd155877c0d23a43 P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6d0487c1f6145f0aa0e09ed071cb3c3 in term 0.
I20260811 15:08:27.641734 1819 leader_election.cc:302] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5680e80a3cf4cfa8e872f64f6189066, b6d0487c1f6145f0aa0e09ed071cb3c3; no voters:
I20260811 15:08:27.641907 2180 raft_consensus.cc:2781] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:08:27.641959 2180 raft_consensus.cc:482] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:08:27.641999 2180 raft_consensus.cc:3037] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:27.643433 2180 raft_consensus.cc:504] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:27.643878 2180 leader_election.cc:288] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [CANDIDATE]: Term 1 election: Requested vote from peers 047b159c89f543b185fc7da705f3eac5 (127.30.8.1:33327), a5680e80a3cf4cfa8e872f64f6189066 (127.30.8.2:44741)
I20260811 15:08:27.644333 1458 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94d10407d26f4739bd155877c0d23a43" candidate_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047b159c89f543b185fc7da705f3eac5"
I20260811 15:08:27.644335 1692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "94d10407d26f4739bd155877c0d23a43" candidate_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5680e80a3cf4cfa8e872f64f6189066"
I20260811 15:08:27.644505 1458 raft_consensus.cc:3037] T 94d10407d26f4739bd155877c0d23a43 P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:27.644505 1692 raft_consensus.cc:3037] T 94d10407d26f4739bd155877c0d23a43 P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:27.646099 1458 raft_consensus.cc:2454] T 94d10407d26f4739bd155877c0d23a43 P 047b159c89f543b185fc7da705f3eac5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6d0487c1f6145f0aa0e09ed071cb3c3 in term 1.
I20260811 15:08:27.646099 1692 raft_consensus.cc:2454] T 94d10407d26f4739bd155877c0d23a43 P a5680e80a3cf4cfa8e872f64f6189066 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6d0487c1f6145f0aa0e09ed071cb3c3 in term 1.
I20260811 15:08:27.646462 1820 leader_election.cc:302] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047b159c89f543b185fc7da705f3eac5, b6d0487c1f6145f0aa0e09ed071cb3c3; no voters:
I20260811 15:08:27.646754 2180 raft_consensus.cc:2781] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:27.646997 2180 raft_consensus.cc:686] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 LEADER]: Becoming Leader. State: Replica: b6d0487c1f6145f0aa0e09ed071cb3c3, State: Running, Role: LEADER
I20260811 15:08:27.647172 2180 consensus_queue.cc:237] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:27.649475 1193 catalog_manager.cc:5495] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6d0487c1f6145f0aa0e09ed071cb3c3 (127.30.8.3). New cstate: current_term: 1 leader_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } health_report { overall_health: HEALTHY } } }
I20260811 15:08:27.670982 2234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:27.667793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49282 (local address 127.30.8.3:33441)
0811 15:08:27.668129 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:27.668137 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:27.668148 (+ 11us) server_negotiation.cc:407] Connection header received
0811 15:08:27.668188 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:27.668191 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:27.668235 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:27.668311 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:27.668558 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:27.669095 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:27.669926 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:27.670159 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:27.670246 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:27.670481 (+ 235us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:27.670559 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:27.670620 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:27.670725 (+ 105us) server_negotiation.cc:299] Negotiation successful
0811 15:08:27.670782 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":258,"thread_start_us":152,"threads_started":1}
I20260811 15:08:27.677043 1692 raft_consensus.cc:1261] T 94d10407d26f4739bd155877c0d23a43 P a5680e80a3cf4cfa8e872f64f6189066 [term 1 FOLLOWER]: Refusing update from remote peer b6d0487c1f6145f0aa0e09ed071cb3c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:27.677101 1458 raft_consensus.cc:1261] T 94d10407d26f4739bd155877c0d23a43 P 047b159c89f543b185fc7da705f3eac5 [term 1 FOLLOWER]: Refusing update from remote peer b6d0487c1f6145f0aa0e09ed071cb3c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:27.677484 2232 consensus_queue.cc:1035] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:27.677663 2159 consensus_queue.cc:1035] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:27.680989 2164 mvcc.cc:205] Tried to move back new op lower bound from 7317343877842034688 to 7317343877724692480. Current Snapshot: MvccSnapshot[applied={T|T < 7317343877842034688}]
I20260811 15:08:27.706295 1195 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:27.706462 1195 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:27.708566 1195 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365: Sending DeleteTablet for 3 replicas of tablet 94d10407d26f4739bd155877c0d23a43
I20260811 15:08:27.709154 1425 tablet_service.cc:1514] Processing DeleteTablet for tablet 94d10407d26f4739bd155877c0d23a43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:27 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:43890
I20260811 15:08:27.709306 1664 tablet_service.cc:1514] Processing DeleteTablet for tablet 94d10407d26f4739bd155877c0d23a43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:27 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:55802
I20260811 15:08:27.709364 2190 tablet_replica.cc:330] T 94d10407d26f4739bd155877c0d23a43 P 047b159c89f543b185fc7da705f3eac5: stopping tablet replica
I20260811 15:08:27.709448 1897 tablet_service.cc:1514] Processing DeleteTablet for tablet 94d10407d26f4739bd155877c0d23a43 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:27 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:49256
I20260811 15:08:27.709481 2189 tablet_replica.cc:330] T 94d10407d26f4739bd155877c0d23a43 P a5680e80a3cf4cfa8e872f64f6189066: stopping tablet replica
I20260811 15:08:27.709501 2190 raft_consensus.cc:2229] T 94d10407d26f4739bd155877c0d23a43 P 047b159c89f543b185fc7da705f3eac5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:27.709616 2189 raft_consensus.cc:2229] T 94d10407d26f4739bd155877c0d23a43 P a5680e80a3cf4cfa8e872f64f6189066 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:27.709626 2190 raft_consensus.cc:2258] T 94d10407d26f4739bd155877c0d23a43 P 047b159c89f543b185fc7da705f3eac5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:27.709642 2192 tablet_replica.cc:330] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3: stopping tablet replica
I20260811 15:08:27.709743 2189 raft_consensus.cc:2258] T 94d10407d26f4739bd155877c0d23a43 P a5680e80a3cf4cfa8e872f64f6189066 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:27.709743 2192 raft_consensus.cc:2229] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:08:27.709944 2192 raft_consensus.cc:2258] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:27.710412 2189 ts_tablet_manager.cc:1905] T 94d10407d26f4739bd155877c0d23a43 P a5680e80a3cf4cfa8e872f64f6189066: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:27.710412 1195 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 15:08:27.710705 2192 ts_tablet_manager.cc:1905] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:27.710757 2190 ts_tablet_manager.cc:1905] T 94d10407d26f4739bd155877c0d23a43 P 047b159c89f543b185fc7da705f3eac5: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 15:08:27.710841 1195 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:08:27.713157 2189 ts_tablet_manager.cc:1918] T 94d10407d26f4739bd155877c0d23a43 P a5680e80a3cf4cfa8e872f64f6189066: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 15:08:27.713217 2189 log.cc:1196] T 94d10407d26f4739bd155877c0d23a43 P a5680e80a3cf4cfa8e872f64f6189066: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/94d10407d26f4739bd155877c0d23a43
I20260811 15:08:27.713238 2192 ts_tablet_manager.cc:1918] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 15:08:27.713238 2190 ts_tablet_manager.cc:1918] T 94d10407d26f4739bd155877c0d23a43 P 047b159c89f543b185fc7da705f3eac5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 15:08:27.713311 2192 log.cc:1196] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/94d10407d26f4739bd155877c0d23a43
I20260811 15:08:27.713311 2190 log.cc:1196] T 94d10407d26f4739bd155877c0d23a43 P 047b159c89f543b185fc7da705f3eac5: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-0/wal/wals/94d10407d26f4739bd155877c0d23a43
I20260811 15:08:27.713389 2189 ts_tablet_manager.cc:1939] T 94d10407d26f4739bd155877c0d23a43 P a5680e80a3cf4cfa8e872f64f6189066: Deleting consensus metadata
I20260811 15:08:27.713562 2192 ts_tablet_manager.cc:1939] T 94d10407d26f4739bd155877c0d23a43 P b6d0487c1f6145f0aa0e09ed071cb3c3: Deleting consensus metadata
I20260811 15:08:27.713577 2190 ts_tablet_manager.cc:1939] T 94d10407d26f4739bd155877c0d23a43 P 047b159c89f543b185fc7da705f3eac5: Deleting consensus metadata
I20260811 15:08:27.714263 1177 catalog_manager.cc:4841] TS 047b159c89f543b185fc7da705f3eac5 (127.30.8.1:33327): tablet 94d10407d26f4739bd155877c0d23a43 (table test-table [id=27d2122bc696488296cb6519cb0b96c8]) successfully deleted
I20260811 15:08:27.714381 1176 catalog_manager.cc:4841] TS b6d0487c1f6145f0aa0e09ed071cb3c3 (127.30.8.3:33441): tablet 94d10407d26f4739bd155877c0d23a43 (table test-table [id=27d2122bc696488296cb6519cb0b96c8]) successfully deleted
I20260811 15:08:27.714637 1176 catalog_manager.cc:4841] TS a5680e80a3cf4cfa8e872f64f6189066 (127.30.8.2:44741): tablet 94d10407d26f4739bd155877c0d23a43 (table test-table [id=27d2122bc696488296cb6519cb0b96c8]) successfully deleted
I20260811 15:08:27.717170 1425 tablet_service.cc:1467] Processing CreateTablet for tablet 7e4dd0fdd6f8454b9ceb285a45a6551a (DEFAULT_TABLE table=test-table [id=160dc55fa6e147859b61983a2ef9bbc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:27.717208 1664 tablet_service.cc:1467] Processing CreateTablet for tablet 7e4dd0fdd6f8454b9ceb285a45a6551a (DEFAULT_TABLE table=test-table [id=160dc55fa6e147859b61983a2ef9bbc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:27.717240 1897 tablet_service.cc:1467] Processing CreateTablet for tablet 7e4dd0fdd6f8454b9ceb285a45a6551a (DEFAULT_TABLE table=test-table [id=160dc55fa6e147859b61983a2ef9bbc9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:27.717476 1425 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e4dd0fdd6f8454b9ceb285a45a6551a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:27.717476 1897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e4dd0fdd6f8454b9ceb285a45a6551a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:27.717486 1664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e4dd0fdd6f8454b9ceb285a45a6551a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:27.719465 2221 tablet_bootstrap.cc:492] T 7e4dd0fdd6f8454b9ceb285a45a6551a P a5680e80a3cf4cfa8e872f64f6189066: Bootstrap starting.
I20260811 15:08:27.719988 2220 tablet_bootstrap.cc:492] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3: Bootstrap starting.
I20260811 15:08:27.720396 2221 tablet_bootstrap.cc:654] T 7e4dd0fdd6f8454b9ceb285a45a6551a P a5680e80a3cf4cfa8e872f64f6189066: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:27.720777 2220 tablet_bootstrap.cc:654] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:27.720932 2218 tablet_bootstrap.cc:492] T 7e4dd0fdd6f8454b9ceb285a45a6551a P 047b159c89f543b185fc7da705f3eac5: Bootstrap starting.
I20260811 15:08:27.721817 2218 tablet_bootstrap.cc:654] T 7e4dd0fdd6f8454b9ceb285a45a6551a P 047b159c89f543b185fc7da705f3eac5: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:27.722726 2221 tablet_bootstrap.cc:492] T 7e4dd0fdd6f8454b9ceb285a45a6551a P a5680e80a3cf4cfa8e872f64f6189066: No bootstrap required, opened a new log
I20260811 15:08:27.722826 2221 ts_tablet_manager.cc:1397] T 7e4dd0fdd6f8454b9ceb285a45a6551a P a5680e80a3cf4cfa8e872f64f6189066: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 15:08:27.723274 2221 raft_consensus.cc:348] T 7e4dd0fdd6f8454b9ceb285a45a6551a P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:27.723381 2221 raft_consensus.cc:374] T 7e4dd0fdd6f8454b9ceb285a45a6551a P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:27.723402 2221 raft_consensus.cc:729] T 7e4dd0fdd6f8454b9ceb285a45a6551a P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a5680e80a3cf4cfa8e872f64f6189066, State: Initialized, Role: FOLLOWER
I20260811 15:08:27.723517 2221 consensus_queue.cc:260] T 7e4dd0fdd6f8454b9ceb285a45a6551a P a5680e80a3cf4cfa8e872f64f6189066 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:27.723759 2221 ts_tablet_manager.cc:1428] T 7e4dd0fdd6f8454b9ceb285a45a6551a P a5680e80a3cf4cfa8e872f64f6189066: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:27.729830 2220 tablet_bootstrap.cc:492] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3: No bootstrap required, opened a new log
I20260811 15:08:27.729830 2218 tablet_bootstrap.cc:492] T 7e4dd0fdd6f8454b9ceb285a45a6551a P 047b159c89f543b185fc7da705f3eac5: No bootstrap required, opened a new log
I20260811 15:08:27.729970 2220 ts_tablet_manager.cc:1397] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3: Time spent bootstrapping tablet: real 0.010s user 0.001s sys 0.000s
I20260811 15:08:27.729970 2218 ts_tablet_manager.cc:1397] T 7e4dd0fdd6f8454b9ceb285a45a6551a P 047b159c89f543b185fc7da705f3eac5: Time spent bootstrapping tablet: real 0.009s user 0.001s sys 0.000s
I20260811 15:08:27.730455 2218 raft_consensus.cc:348] T 7e4dd0fdd6f8454b9ceb285a45a6551a P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:27.730455 2220 raft_consensus.cc:348] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:27.730597 2220 raft_consensus.cc:374] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:27.730597 2218 raft_consensus.cc:374] T 7e4dd0fdd6f8454b9ceb285a45a6551a P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:27.730631 2218 raft_consensus.cc:729] T 7e4dd0fdd6f8454b9ceb285a45a6551a P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 047b159c89f543b185fc7da705f3eac5, State: Initialized, Role: FOLLOWER
I20260811 15:08:27.730631 2220 raft_consensus.cc:729] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6d0487c1f6145f0aa0e09ed071cb3c3, State: Initialized, Role: FOLLOWER
I20260811 15:08:27.730769 2220 consensus_queue.cc:260] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:27.730769 2218 consensus_queue.cc:260] T 7e4dd0fdd6f8454b9ceb285a45a6551a P 047b159c89f543b185fc7da705f3eac5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:27.731077 2220 ts_tablet_manager.cc:1428] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 15:08:27.731125 2218 ts_tablet_manager.cc:1428] T 7e4dd0fdd6f8454b9ceb285a45a6551a P 047b159c89f543b185fc7da705f3eac5: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 15:08:27.800428 2180 raft_consensus.cc:482] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:08:27.800680 2180 raft_consensus.cc:504] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:27.801249 2180 leader_election.cc:288] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 047b159c89f543b185fc7da705f3eac5 (127.30.8.1:33327), a5680e80a3cf4cfa8e872f64f6189066 (127.30.8.2:44741)
I20260811 15:08:27.801769 1460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e4dd0fdd6f8454b9ceb285a45a6551a" candidate_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047b159c89f543b185fc7da705f3eac5" is_pre_election: true
I20260811 15:08:27.801914 1692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e4dd0fdd6f8454b9ceb285a45a6551a" candidate_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5680e80a3cf4cfa8e872f64f6189066" is_pre_election: true
I20260811 15:08:27.802004 1460 raft_consensus.cc:2454] T 7e4dd0fdd6f8454b9ceb285a45a6551a P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6d0487c1f6145f0aa0e09ed071cb3c3 in term 0.
I20260811 15:08:27.802065 1692 raft_consensus.cc:2454] T 7e4dd0fdd6f8454b9ceb285a45a6551a P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b6d0487c1f6145f0aa0e09ed071cb3c3 in term 0.
I20260811 15:08:27.802402 1820 leader_election.cc:302] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 047b159c89f543b185fc7da705f3eac5, b6d0487c1f6145f0aa0e09ed071cb3c3; no voters:
I20260811 15:08:27.802639 2180 raft_consensus.cc:2781] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:08:27.802733 2180 raft_consensus.cc:482] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:08:27.802764 2180 raft_consensus.cc:3037] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:27.803826 2180 raft_consensus.cc:504] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:27.804229 2180 leader_election.cc:288] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [CANDIDATE]: Term 1 election: Requested vote from peers 047b159c89f543b185fc7da705f3eac5 (127.30.8.1:33327), a5680e80a3cf4cfa8e872f64f6189066 (127.30.8.2:44741)
I20260811 15:08:27.804718 1460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e4dd0fdd6f8454b9ceb285a45a6551a" candidate_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "047b159c89f543b185fc7da705f3eac5"
I20260811 15:08:27.804718 1692 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e4dd0fdd6f8454b9ceb285a45a6551a" candidate_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a5680e80a3cf4cfa8e872f64f6189066"
I20260811 15:08:27.804855 1692 raft_consensus.cc:3037] T 7e4dd0fdd6f8454b9ceb285a45a6551a P a5680e80a3cf4cfa8e872f64f6189066 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:27.804855 1460 raft_consensus.cc:3037] T 7e4dd0fdd6f8454b9ceb285a45a6551a P 047b159c89f543b185fc7da705f3eac5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:27.806028 1460 raft_consensus.cc:2454] T 7e4dd0fdd6f8454b9ceb285a45a6551a P 047b159c89f543b185fc7da705f3eac5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6d0487c1f6145f0aa0e09ed071cb3c3 in term 1.
I20260811 15:08:27.806028 1692 raft_consensus.cc:2454] T 7e4dd0fdd6f8454b9ceb285a45a6551a P a5680e80a3cf4cfa8e872f64f6189066 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b6d0487c1f6145f0aa0e09ed071cb3c3 in term 1.
I20260811 15:08:27.806396 1819 leader_election.cc:302] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a5680e80a3cf4cfa8e872f64f6189066, b6d0487c1f6145f0aa0e09ed071cb3c3; no voters:
I20260811 15:08:27.806587 2180 raft_consensus.cc:2781] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:27.806694 2180 raft_consensus.cc:686] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 LEADER]: Becoming Leader. State: Replica: b6d0487c1f6145f0aa0e09ed071cb3c3, State: Running, Role: LEADER
I20260811 15:08:27.806845 2180 consensus_queue.cc:237] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } }
I20260811 15:08:27.808728 1194 catalog_manager.cc:5495] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 reported cstate change: term changed from 0 to 1, leader changed from <none> to b6d0487c1f6145f0aa0e09ed071cb3c3 (127.30.8.3). New cstate: current_term: 1 leader_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b6d0487c1f6145f0aa0e09ed071cb3c3" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 33441 } health_report { overall_health: HEALTHY } } }
I20260811 15:08:27.838187 1692 raft_consensus.cc:1261] T 7e4dd0fdd6f8454b9ceb285a45a6551a P a5680e80a3cf4cfa8e872f64f6189066 [term 1 FOLLOWER]: Refusing update from remote peer b6d0487c1f6145f0aa0e09ed071cb3c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:27.838212 1460 raft_consensus.cc:1261] T 7e4dd0fdd6f8454b9ceb285a45a6551a P 047b159c89f543b185fc7da705f3eac5 [term 1 FOLLOWER]: Refusing update from remote peer b6d0487c1f6145f0aa0e09ed071cb3c3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:27.838572 2180 consensus_queue.cc:1035] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a5680e80a3cf4cfa8e872f64f6189066" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:27.838801 2057 consensus_queue.cc:1035] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "047b159c89f543b185fc7da705f3eac5" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 33327 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 15:08:27.852699 2154 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 15:08:29.406437 1195 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:29.406607 1195 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:29.409226 1195 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1e2eeb347cb845fa9536f56ef58c7365: Sending DeleteTablet for 3 replicas of tablet 7e4dd0fdd6f8454b9ceb285a45a6551a
I20260811 15:08:29.409870 1664 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e4dd0fdd6f8454b9ceb285a45a6551a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:29 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:55802
I20260811 15:08:29.409873 1425 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e4dd0fdd6f8454b9ceb285a45a6551a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:29 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:43890
I20260811 15:08:29.410247 2282 tablet_replica.cc:330] T 7e4dd0fdd6f8454b9ceb285a45a6551a P a5680e80a3cf4cfa8e872f64f6189066: stopping tablet replica
I20260811 15:08:29.410244 1897 tablet_service.cc:1514] Processing DeleteTablet for tablet 7e4dd0fdd6f8454b9ceb285a45a6551a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:29 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:49256
I20260811 15:08:29.410420 2282 raft_consensus.cc:2229] T 7e4dd0fdd6f8454b9ceb285a45a6551a P a5680e80a3cf4cfa8e872f64f6189066 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:29.410614 2283 tablet_replica.cc:330] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3: stopping tablet replica
I20260811 15:08:29.410619 2281 tablet_replica.cc:330] T 7e4dd0fdd6f8454b9ceb285a45a6551a P 047b159c89f543b185fc7da705f3eac5: stopping tablet replica
I20260811 15:08:29.410616 2282 raft_consensus.cc:2258] T 7e4dd0fdd6f8454b9ceb285a45a6551a P a5680e80a3cf4cfa8e872f64f6189066 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:29.410744 2281 raft_consensus.cc:2229] T 7e4dd0fdd6f8454b9ceb285a45a6551a P 047b159c89f543b185fc7da705f3eac5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:29.410763 2283 raft_consensus.cc:2229] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:08:29.410893 2281 raft_consensus.cc:2258] T 7e4dd0fdd6f8454b9ceb285a45a6551a P 047b159c89f543b185fc7da705f3eac5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:29.411006 2283 raft_consensus.cc:2258] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:29.410274 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 1313
I20260811 15:08:29.422438 2283 ts_tablet_manager.cc:1905] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:29.426327 2282 ts_tablet_manager.cc:1905] T 7e4dd0fdd6f8454b9ceb285a45a6551a P a5680e80a3cf4cfa8e872f64f6189066: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:29.426604 2283 ts_tablet_manager.cc:1918] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 15:08:29.426699 2283 log.cc:1196] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/7e4dd0fdd6f8454b9ceb285a45a6551a
I20260811 15:08:29.427017 2283 ts_tablet_manager.cc:1939] T 7e4dd0fdd6f8454b9ceb285a45a6551a P b6d0487c1f6145f0aa0e09ed071cb3c3: Deleting consensus metadata
I20260811 15:08:29.427906 1176 catalog_manager.cc:4841] TS b6d0487c1f6145f0aa0e09ed071cb3c3 (127.30.8.3:33441): tablet 7e4dd0fdd6f8454b9ceb285a45a6551a (table test-table [id=160dc55fa6e147859b61983a2ef9bbc9]) successfully deleted
I20260811 15:08:29.428772 2282 ts_tablet_manager.cc:1918] T 7e4dd0fdd6f8454b9ceb285a45a6551a P a5680e80a3cf4cfa8e872f64f6189066: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 15:08:29.428848 2282 log.cc:1196] T 7e4dd0fdd6f8454b9ceb285a45a6551a P a5680e80a3cf4cfa8e872f64f6189066: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/7e4dd0fdd6f8454b9ceb285a45a6551a
I20260811 15:08:29.429061 2282 ts_tablet_manager.cc:1939] T 7e4dd0fdd6f8454b9ceb285a45a6551a P a5680e80a3cf4cfa8e872f64f6189066: Deleting consensus metadata
I20260811 15:08:29.429760 1176 catalog_manager.cc:4841] TS a5680e80a3cf4cfa8e872f64f6189066 (127.30.8.2:44741): tablet 7e4dd0fdd6f8454b9ceb285a45a6551a (table test-table [id=160dc55fa6e147859b61983a2ef9bbc9]) successfully deleted
W20260811 15:08:29.432102 1177 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.8.1:33327 (error 108)
I20260811 15:08:29.432979 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 1553
I20260811 15:08:29.441188 2290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:29.432833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:33327 (local address 127.0.0.1:43904)
0811 15:08:29.440769 (+ 7936us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:29.440998 (+ 229us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.1:33327: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":97,"threads_started":1}
W20260811 15:08:29.441505 1177 catalog_manager.cc:4584] TS 047b159c89f543b185fc7da705f3eac5 (127.30.8.1:33327): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7e4dd0fdd6f8454b9ceb285a45a6551a: Network error: Client connection negotiation failed: client connection to 127.30.8.1:33327: connect: Connection refused (error 111)
I20260811 15:08:29.450214 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 1792
I20260811 15:08:29.472268 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 1147
2026-08-11T15:08:29Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7196 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:08:29 dist-test-slave-1vmg krb5kdc[2297](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:08:29 dist-test-slave-1vmg krb5kdc[2297](info): set up 2 sockets
Aug 11 15:08:29 dist-test-slave-1vmg krb5kdc[2297](info): commencing operation
krb5kdc: starting...
W20260811 15:08:30.676733 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:08:30 dist-test-slave-1vmg krb5kdc[2297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:08:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:08:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:08:30.810376 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:44735
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:34751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:44735
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:30.930274 2318 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:30.930603 2318 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:30.930665 2318 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:30.934839 2318 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:08:30.934964 2318 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:30.934995 2318 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:30.935019 2318 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:08:30.935050 2318 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:08:30.940845 2318 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:34751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:44735
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:44735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2318
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:30.942121 2318 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:30.943274 2318 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:30.952543 2329 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:30.952580 2328 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:30.952543 2331 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:30.952663 2318 server_base.cc:1034] running on GCE node
I20260811 15:08:30.953425 2318 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:30.954653 2318 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:30.955853 2318 hybrid_clock.cc:648] HybridClock initialized: now 1786460910955814 us; error 50 us; skew 500 ppm
Aug 11 15:08:30 dist-test-slave-1vmg krb5kdc[2297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460910, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:30.958734 2318 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:08:30.959873 2318 webserver.cc:460] Webserver started at http://127.30.8.62:43073/ using document root <none> and password file <none>
I20260811 15:08:30.960470 2318 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:30.960538 2318 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:30.960752 2318 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:30.963033 2318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "4f82aa02fa934d50965fcf9cf07ef2cb"
format_stamp: "Formatted at 2026-08-11 15:08:30 on dist-test-slave-1vmg"
server_key: "6b1e514e5d4734bcd7da5ed1136f6157"
server_key_iv: "9e16a58ed2d3d05736f6e2d6585674c3"
server_key_version: "encryptionkey@0"
I20260811 15:08:30.963629 2318 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "4f82aa02fa934d50965fcf9cf07ef2cb"
format_stamp: "Formatted at 2026-08-11 15:08:30 on dist-test-slave-1vmg"
server_key: "6b1e514e5d4734bcd7da5ed1136f6157"
server_key_iv: "9e16a58ed2d3d05736f6e2d6585674c3"
server_key_version: "encryptionkey@0"
I20260811 15:08:30.967868 2318 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 15:08:30.970440 2342 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:30.971783 2318 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 15:08:30.971908 2318 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "4f82aa02fa934d50965fcf9cf07ef2cb"
format_stamp: "Formatted at 2026-08-11 15:08:30 on dist-test-slave-1vmg"
server_key: "6b1e514e5d4734bcd7da5ed1136f6157"
server_key_iv: "9e16a58ed2d3d05736f6e2d6585674c3"
server_key_version: "encryptionkey@0"
I20260811 15:08:30.972034 2318 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:30.986493 2318 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:30.990010 2318 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:30.990212 2318 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:30.997618 2318 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:44735
I20260811 15:08:30.997633 2432 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:44735 every 8 connection(s)
I20260811 15:08:30.998677 2318 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:08:31.001740 2434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:31.007642 2434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb: Bootstrap starting.
I20260811 15:08:31.008579 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 2318
I20260811 15:08:31.008687 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:08:31.008991 30752 external_mini_cluster.cc:1428] Setting key 41347b64776d1e96fdf074fb39454b7d
I20260811 15:08:31.010679 2434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:31.011492 2434 log.cc:824] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:31.013461 2434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb: No bootstrap required, opened a new log
I20260811 15:08:31.017060 2434 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f82aa02fa934d50965fcf9cf07ef2cb" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44735 } }
I20260811 15:08:31.017323 2434 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:31.017382 2434 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f82aa02fa934d50965fcf9cf07ef2cb, State: Initialized, Role: FOLLOWER
Aug 11 15:08:31 dist-test-slave-1vmg krb5kdc[2297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:31.017983 2434 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f82aa02fa934d50965fcf9cf07ef2cb" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44735 } }
I20260811 15:08:31.018185 2434 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:08:31.018261 2434 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:08:31.018365 2434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:31.019392 2434 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f82aa02fa934d50965fcf9cf07ef2cb" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44735 } }
I20260811 15:08:31.019775 2434 leader_election.cc:302] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f82aa02fa934d50965fcf9cf07ef2cb; no voters:
I20260811 15:08:31.020128 2434 leader_election.cc:288] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:08:31.020223 2441 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:31.020450 2441 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb [term 1 LEADER]: Becoming Leader. State: Replica: 4f82aa02fa934d50965fcf9cf07ef2cb, State: Running, Role: LEADER
I20260811 15:08:31.020829 2441 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f82aa02fa934d50965fcf9cf07ef2cb" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44735 } }
I20260811 15:08:31.021557 2434 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:08:31.022960 2442 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f82aa02fa934d50965fcf9cf07ef2cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f82aa02fa934d50965fcf9cf07ef2cb" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44735 } } }
I20260811 15:08:31.023090 2442 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:31.023553 2438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:31.010200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39842 (local address 127.30.8.62:44735)
0811 15:08:31.010667 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:31.010684 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:31.010719 (+ 35us) server_negotiation.cc:407] Connection header received
0811 15:08:31.011455 (+ 736us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:31.011475 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:31.011851 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:31.012188 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:31.012995 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:31.014030 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:31.014890 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:31.015440 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:31.015595 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:31.018227 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:31.018234 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:31.018254 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:31.018291 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:31.020196 (+ 1905us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:31.020705 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:31.020711 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:31.020717 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:31.020850 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:31.021146 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:31.021149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:31.021151 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:31.021642 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:31.021840 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:31.022131 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 15:08:31.022460 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":141,"threads_started":1}
I20260811 15:08:31.023504 2444 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f82aa02fa934d50965fcf9cf07ef2cb. Latest consensus state: current_term: 1 leader_uuid: "4f82aa02fa934d50965fcf9cf07ef2cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f82aa02fa934d50965fcf9cf07ef2cb" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44735 } } }
I20260811 15:08:31.024446 2444 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb [sys.catalog]: This master's current role is: LEADER
W20260811 15:08:31.026805 2468 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:08:31.026872 2468 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:08:31.026973 2454 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:08:31.027938 2454 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:08:31.034092 2454 catalog_manager.cc:1347] Generated new cluster ID: d4a95b89a669434bb5deb92363edfe95
I20260811 15:08:31.034272 2454 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:08:31.057542 2454 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:08:31.058794 2454 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:08:31.063795 2454 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f82aa02fa934d50965fcf9cf07ef2cb: Generated new TSK 0
I20260811 15:08:31.064523 2454 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
I20260811 15:08:31.101852 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:44735
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:34751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:31.246104 2479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:31.246440 2479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:31.246487 2479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:31.250728 2479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:31.250813 2479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:31.250877 2479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:08:31.256572 2479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:34751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:44735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:31.257798 2479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:31.258860 2479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:31.269259 2486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:31.269245 2488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:31.269235 2485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:31.269547 2479 server_base.cc:1034] running on GCE node
I20260811 15:08:31.270042 2479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:31.270769 2479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:31.271941 2479 hybrid_clock.cc:648] HybridClock initialized: now 1786460911271914 us; error 53 us; skew 500 ppm
Aug 11 15:08:31 dist-test-slave-1vmg krb5kdc[2297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:31.274895 2479 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:08:31.276027 2479 webserver.cc:460] Webserver started at http://127.30.8.1:38313/ using document root <none> and password file <none>
I20260811 15:08:31.276648 2479 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:31.276721 2479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:31.276952 2479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:31.279310 2479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "237b9b48bae947fba52c434b906a3a0d"
format_stamp: "Formatted at 2026-08-11 15:08:31 on dist-test-slave-1vmg"
server_key: "bf17590ec6284e75fb3773744fc944f7"
server_key_iv: "caadc963cea9ec569416b75add0d862b"
server_key_version: "encryptionkey@0"
I20260811 15:08:31.279928 2479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "237b9b48bae947fba52c434b906a3a0d"
format_stamp: "Formatted at 2026-08-11 15:08:31 on dist-test-slave-1vmg"
server_key: "bf17590ec6284e75fb3773744fc944f7"
server_key_iv: "caadc963cea9ec569416b75add0d862b"
server_key_version: "encryptionkey@0"
I20260811 15:08:31.284108 2479 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:08:31.286886 2501 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:31.288602 2479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 15:08:31.288743 2479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "237b9b48bae947fba52c434b906a3a0d"
format_stamp: "Formatted at 2026-08-11 15:08:31 on dist-test-slave-1vmg"
server_key: "bf17590ec6284e75fb3773744fc944f7"
server_key_iv: "caadc963cea9ec569416b75add0d862b"
server_key_version: "encryptionkey@0"
I20260811 15:08:31.288861 2479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:31.318614 2479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:31.322177 2479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:31.322374 2479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:31.323105 2479 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:31.324074 2479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:31.324138 2479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:31.324201 2479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:31.324229 2479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:31.336262 2479 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:33531
I20260811 15:08:31.336292 2712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:33531 every 8 connection(s)
I20260811 15:08:31.337287 2479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:08:31.341492 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 2479
I20260811 15:08:31.341630 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:08:31.341991 30752 external_mini_cluster.cc:1428] Setting key 953d7324ec02645fd11d595e65e36edd
Aug 11 15:08:31 dist-test-slave-1vmg krb5kdc[2297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.8.2@KRBTEST.COM; defaulting to no policy
I20260811 15:08:31.354907 2438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:31.339381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:42245 (local address 127.30.8.62:44735)
0811 15:08:31.339548 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:31.339554 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:31.340523 (+ 969us) server_negotiation.cc:407] Connection header received
0811 15:08:31.342184 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:31.342187 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:31.342268 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:31.342344 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:31.343935 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:31.344465 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:31.346095 (+ 1630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:31.346344 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:31.346452 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:31.349934 (+ 3482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:31.349939 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:31.349942 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:31.349984 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:31.351707 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:31.352656 (+ 949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:31.352659 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:31.352661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:31.352773 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:31.353477 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:31.353481 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:31.353482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:31.353716 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:31.353812 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:31.354582 (+ 770us) server_negotiation.cc:299] Negotiation successful
0811 15:08:31.354744 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 15:08:31.355949 2717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:31.339826 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:44735 (local address 127.30.8.1:42245)
0811 15:08:31.340344 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:31.340393 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:31.341914 (+ 1521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:31.342564 (+ 650us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:31.342572 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:31.343098 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:31.343775 (+ 677us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:31.343799 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:31.344607 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:31.344612 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:31.345949 (+ 1337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:31.345961 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:31.346512 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:31.346517 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:31.346714 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:31.347534 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:31.347554 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:31.349655 (+ 2101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:31.351891 (+ 2236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:31.351901 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:31.351940 (+ 39us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:31.352403 (+ 463us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:31.352889 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:31.352894 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:31.352897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:31.353243 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:31.353839 (+ 596us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:31.353849 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:31.354302 (+ 453us) client_negotiation.cc:769] Sending connection context
0811 15:08:31.354551 (+ 249us) client_negotiation.cc:240] Negotiation successful
0811 15:08:31.355033 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":132,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.2@KRBTEST.COM" created.
I20260811 15:08:31.357772 2713 heartbeater.cc:347] Connected to a master server at 127.30.8.62:44735
I20260811 15:08:31.358121 2713 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:31.358731 2713 heartbeater.cc:511] Master 127.30.8.62:44735 requested a full tablet report, sending...
I20260811 15:08:31.360559 2368 ts_manager.cc:194] Registered new tserver with Master: 237b9b48bae947fba52c434b906a3a0d (127.30.8.1:33531)
I20260811 15:08:31.362088 2368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:42245
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
I20260811 15:08:31.376573 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:44735
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:34751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:31.506235 2720 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:31.506563 2720 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:31.506611 2720 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:31.510735 2720 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:31.510839 2720 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:31.510903 2720 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:08:31.516671 2720 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:34751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:44735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2720
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:31.517894 2720 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:31.518968 2720 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:31.529417 2728 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:31.529428 2733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:31.529428 2729 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:31.529717 2720 server_base.cc:1034] running on GCE node
I20260811 15:08:31.530267 2720 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:31.531062 2720 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:31.532244 2720 hybrid_clock.cc:648] HybridClock initialized: now 1786460911532226 us; error 41 us; skew 500 ppm
Aug 11 15:08:31 dist-test-slave-1vmg krb5kdc[2297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:31.535424 2720 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:08:31.536478 2720 webserver.cc:460] Webserver started at http://127.30.8.2:37767/ using document root <none> and password file <none>
I20260811 15:08:31.537086 2720 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:31.537156 2720 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:31.537359 2720 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:31.539682 2720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "1c5e0a321dae4546ae5849cee8502ca8"
format_stamp: "Formatted at 2026-08-11 15:08:31 on dist-test-slave-1vmg"
server_key: "a8820249049da126de1d4a4b3de2da0c"
server_key_iv: "de5916c6d52eb61ad02e0a7886aa8112"
server_key_version: "encryptionkey@0"
I20260811 15:08:31.540323 2720 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "1c5e0a321dae4546ae5849cee8502ca8"
format_stamp: "Formatted at 2026-08-11 15:08:31 on dist-test-slave-1vmg"
server_key: "a8820249049da126de1d4a4b3de2da0c"
server_key_iv: "de5916c6d52eb61ad02e0a7886aa8112"
server_key_version: "encryptionkey@0"
I20260811 15:08:31.544608 2720 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:08:31.547416 2743 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:31.548767 2720 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:08:31.548915 2720 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "1c5e0a321dae4546ae5849cee8502ca8"
format_stamp: "Formatted at 2026-08-11 15:08:31 on dist-test-slave-1vmg"
server_key: "a8820249049da126de1d4a4b3de2da0c"
server_key_iv: "de5916c6d52eb61ad02e0a7886aa8112"
server_key_version: "encryptionkey@0"
I20260811 15:08:31.549074 2720 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:31.573848 2720 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:31.577301 2720 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:31.577529 2720 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:31.578212 2720 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:31.579216 2720 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:31.579279 2720 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:31.579344 2720 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:31.579373 2720 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:31.589301 2720 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:34687
I20260811 15:08:31.589319 2953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:34687 every 8 connection(s)
I20260811 15:08:31.590363 2720 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:08:31.595821 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 2720
I20260811 15:08:31.595943 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:08:31.596192 30752 external_mini_cluster.cc:1428] Setting key 82a828632eb78b0cf437606117c8f026
Aug 11 15:08:31 dist-test-slave-1vmg krb5kdc[2297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:31.606177 2438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:31.592379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:60865 (local address 127.30.8.62:44735)
0811 15:08:31.592554 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:31.592558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:31.593529 (+ 971us) server_negotiation.cc:407] Connection header received
0811 15:08:31.595101 (+ 1572us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:31.595105 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:31.595173 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:31.595266 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:31.596689 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:31.597334 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:31.598643 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:31.598997 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:31.599123 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:31.602044 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:31.602049 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:31.602051 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:31.602082 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:31.603690 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:31.604340 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:31.604345 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:31.604347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:31.604448 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:31.604837 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:31.604841 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:31.604843 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:31.605085 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:31.605192 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:31.605808 (+ 616us) server_negotiation.cc:299] Negotiation successful
0811 15:08:31.605969 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":19,"server-negotiator.queue_time_us":63}
WARNING: no policy specified for kudu/127.30.8.3@KRBTEST.COM; defaulting to no policy
I20260811 15:08:31.607204 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:31.592800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:44735 (local address 127.30.8.2:60865)
0811 15:08:31.593353 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:31.593401 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:31.594803 (+ 1402us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:31.595480 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:31.595497 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:31.595909 (+ 412us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:31.596522 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:31.596542 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:31.597495 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:31.597499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:31.598493 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:31.598506 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:31.599158 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:31.599161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:31.599329 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:31.600009 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:31.600030 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:31.601809 (+ 1779us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:31.603844 (+ 2035us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:31.603851 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:31.603870 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:31.604215 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:31.604552 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:31.604556 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:31.604557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:31.604733 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:31.605202 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:31.605211 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:31.605526 (+ 315us) client_negotiation.cc:769] Sending connection context
0811 15:08:31.605744 (+ 218us) client_negotiation.cc:240] Negotiation successful
0811 15:08:31.606165 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":139,"threads_started":1}
I20260811 15:08:31.608532 2956 heartbeater.cc:347] Connected to a master server at 127.30.8.62:44735
I20260811 15:08:31.608810 2956 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:31.609400 2956 heartbeater.cc:511] Master 127.30.8.62:44735 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.3@KRBTEST.COM" created.
I20260811 15:08:31.610708 2368 ts_manager.cc:194] Registered new tserver with Master: 1c5e0a321dae4546ae5849cee8502ca8 (127.30.8.2:34687)
I20260811 15:08:31.611351 2368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:60865
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
I20260811 15:08:31.625768 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:44735
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:34751
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:31.749452 2980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:31.749747 2980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:31.749800 2980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:31.754081 2980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:31.754172 2980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:31.754256 2980 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:08:31.760087 2980 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:34751
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:44735
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2980
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:31.761402 2980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:31.762553 2980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:31.772882 2997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:31.772907 2990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:31.773103 2980 server_base.cc:1034] running on GCE node
W20260811 15:08:31.772897 2989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:31.773770 2980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:31.774523 2980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:31.775687 2980 hybrid_clock.cc:648] HybridClock initialized: now 1786460911775671 us; error 33 us; skew 500 ppm
Aug 11 15:08:31 dist-test-slave-1vmg krb5kdc[2297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:31.778692 2980 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:08:31.779804 2980 webserver.cc:460] Webserver started at http://127.30.8.3:45245/ using document root <none> and password file <none>
I20260811 15:08:31.780408 2980 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:31.780473 2980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:31.780675 2980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:31.782886 2980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "b7456de677a644fbb38f5b488cbc94df"
format_stamp: "Formatted at 2026-08-11 15:08:31 on dist-test-slave-1vmg"
server_key: "d8c59fb5e6d776be36d88b35456bd3a9"
server_key_iv: "cd1ff6e103ce99ba0a2a391b41bd0bb4"
server_key_version: "encryptionkey@0"
I20260811 15:08:31.783556 2980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "b7456de677a644fbb38f5b488cbc94df"
format_stamp: "Formatted at 2026-08-11 15:08:31 on dist-test-slave-1vmg"
server_key: "d8c59fb5e6d776be36d88b35456bd3a9"
server_key_iv: "cd1ff6e103ce99ba0a2a391b41bd0bb4"
server_key_version: "encryptionkey@0"
I20260811 15:08:31.787602 2980 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:08:31.790256 3005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:31.791599 2980 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:08:31.791729 2980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "b7456de677a644fbb38f5b488cbc94df"
format_stamp: "Formatted at 2026-08-11 15:08:31 on dist-test-slave-1vmg"
server_key: "d8c59fb5e6d776be36d88b35456bd3a9"
server_key_iv: "cd1ff6e103ce99ba0a2a391b41bd0bb4"
server_key_version: "encryptionkey@0"
I20260811 15:08:31.791843 2980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:31.806905 2980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:31.810444 2980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:31.810654 2980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:31.811363 2980 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:31.812292 2980 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:31.812345 2980 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:31.812403 2980 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:31.812429 2980 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:31.823458 2980 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:38145
I20260811 15:08:31.823479 3204 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:38145 every 8 connection(s)
I20260811 15:08:31.824553 2980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:08:31.834086 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 2980
I20260811 15:08:31.834219 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:08:31.834546 30752 external_mini_cluster.cc:1428] Setting key f2efb59fccfd5c941cf2a11f6f41f983
Aug 11 15:08:31 dist-test-slave-1vmg krb5kdc[2297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:31.839262 2438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:31.826526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:45297 (local address 127.30.8.62:44735)
0811 15:08:31.826659 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:31.826664 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:31.827615 (+ 951us) server_negotiation.cc:407] Connection header received
0811 15:08:31.829032 (+ 1417us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:31.829036 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:31.829101 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:31.829194 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:31.830549 (+ 1355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:31.831136 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:31.832361 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:31.832599 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:31.832700 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:31.835646 (+ 2946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:31.835650 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:31.835653 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:31.835689 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:31.837243 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:31.837750 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:31.837752 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:31.837753 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:31.837838 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:31.838122 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:31.838124 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:31.838125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:31.838324 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:31.838402 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:31.839009 (+ 607us) server_negotiation.cc:299] Negotiation successful
0811 15:08:31.839130 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 15:08:31.840194 3215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:31.826893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:44735 (local address 127.30.8.3:45297)
0811 15:08:31.827449 (+ 556us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:31.827501 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:31.828798 (+ 1297us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:31.829399 (+ 601us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:31.829406 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:31.829831 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:31.830378 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:31.830394 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:31.831273 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:31.831276 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:31.832221 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:31.832233 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:31.832778 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:31.832784 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:31.832964 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:31.833617 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:31.833638 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:31.835454 (+ 1816us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:31.837360 (+ 1906us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:31.837365 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:31.837382 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:31.837668 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:31.837908 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:31.837910 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:31.837912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:31.838053 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:31.838420 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:31.838426 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:31.838744 (+ 318us) client_negotiation.cc:769] Sending connection context
0811 15:08:31.838975 (+ 231us) client_negotiation.cc:240] Negotiation successful
0811 15:08:31.839293 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":132,"threads_started":1}
I20260811 15:08:31.841390 3209 heartbeater.cc:347] Connected to a master server at 127.30.8.62:44735
I20260811 15:08:31.841670 3209 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:31.842247 3209 heartbeater.cc:511] Master 127.30.8.62:44735 requested a full tablet report, sending...
I20260811 15:08:31.843392 2368 ts_manager.cc:194] Registered new tserver with Master: b7456de677a644fbb38f5b488cbc94df (127.30.8.3:38145)
I20260811 15:08:31.843986 2368 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:45297
I20260811 15:08:31.848757 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 15:08:31.865952 2438 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:08:31.864514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39846 (local address 127.30.8.62:44735)
0811 15:08:31.864648 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:31.864652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:31.864748 (+ 96us) server_negotiation.cc:407] Connection header received
0811 15:08:31.864870 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:31.864873 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:31.864921 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:31.865014 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:31.865784 (+ 770us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39846: BlockingRecv error: recv got EOF from 127.0.0.1:39846 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 15:08:31.867594 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 2479
I20260811 15:08:31.919102 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 2720
I20260811 15:08:31.934182 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 2980
I20260811 15:08:31.952186 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 2318
2026-08-11T15:08:31Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2476 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:08:32 dist-test-slave-1vmg krb5kdc[3231](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:08:32 dist-test-slave-1vmg krb5kdc[3231](info): set up 2 sockets
Aug 11 15:08:32 dist-test-slave-1vmg krb5kdc[3231](info): commencing operation
krb5kdc: starting...
W20260811 15:08:33.140990 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:08:33 dist-test-slave-1vmg krb5kdc[3231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:08:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:08:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:08:33.269536 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:40993
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:41181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:40993
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:33.391090 3252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:33.391431 3252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:33.391494 3252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:33.395704 3252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:08:33.395794 3252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:33.395823 3252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:33.395844 3252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:08:33.395864 3252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:08:33.401599 3252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:41181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:40993
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:40993
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:33.402833 3252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:33.403954 3252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:33.412736 3258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:33.412736 3259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:33.412742 3262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:33.412914 3252 server_base.cc:1034] running on GCE node
I20260811 15:08:33.413910 3252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:33.415174 3252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:33.416342 3252 hybrid_clock.cc:648] HybridClock initialized: now 1786460913416320 us; error 38 us; skew 500 ppm
Aug 11 15:08:33 dist-test-slave-1vmg krb5kdc[3231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460913, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:33.419260 3252 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:08:33.420264 3252 webserver.cc:460] Webserver started at http://127.30.8.62:44771/ using document root <none> and password file <none>
I20260811 15:08:33.420863 3252 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:33.420948 3252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:33.421185 3252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:33.423435 3252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "df0a072fdbf345669f77895618e5f20a"
format_stamp: "Formatted at 2026-08-11 15:08:33 on dist-test-slave-1vmg"
server_key: "cba6bc343e467ed8387621ff0c09cfa5"
server_key_iv: "adf9418bde38d12c9cb9285a1ca05f0f"
server_key_version: "encryptionkey@0"
I20260811 15:08:33.424033 3252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "df0a072fdbf345669f77895618e5f20a"
format_stamp: "Formatted at 2026-08-11 15:08:33 on dist-test-slave-1vmg"
server_key: "cba6bc343e467ed8387621ff0c09cfa5"
server_key_iv: "adf9418bde38d12c9cb9285a1ca05f0f"
server_key_version: "encryptionkey@0"
I20260811 15:08:33.428262 3252 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:08:33.430826 3270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:33.432155 3252 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 15:08:33.432292 3252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "df0a072fdbf345669f77895618e5f20a"
format_stamp: "Formatted at 2026-08-11 15:08:33 on dist-test-slave-1vmg"
server_key: "cba6bc343e467ed8387621ff0c09cfa5"
server_key_iv: "adf9418bde38d12c9cb9285a1ca05f0f"
server_key_version: "encryptionkey@0"
I20260811 15:08:33.432410 3252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:33.443638 3252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:33.447119 3252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:33.447325 3252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:33.454439 3252 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:40993
I20260811 15:08:33.454728 3356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:40993 every 8 connection(s)
I20260811 15:08:33.455539 3252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:08:33.456938 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 3252
I20260811 15:08:33.457038 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:08:33.457294 30752 external_mini_cluster.cc:1428] Setting key e18c961e146c54f2125c0bd52623e58f
I20260811 15:08:33.458884 3357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:33.466046 3357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a: Bootstrap starting.
Aug 11 15:08:33 dist-test-slave-1vmg krb5kdc[3231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460913, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:33.468477 3357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:33.469245 3357 log.cc:824] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:33.471467 3357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a: No bootstrap required, opened a new log
I20260811 15:08:33.471630 3361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:33.458600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50988 (local address 127.30.8.62:40993)
0811 15:08:33.459061 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:33.459079 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:33.459116 (+ 37us) server_negotiation.cc:407] Connection header received
0811 15:08:33.459784 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:33.459801 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:33.460142 (+ 341us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:33.460435 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:33.461366 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:33.462369 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:33.463516 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:33.464008 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:33.464129 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:33.466978 (+ 2849us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:33.466985 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:33.467001 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:33.467033 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:33.468622 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:33.469063 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:33.469069 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:33.469075 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:33.469195 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:33.469461 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:33.469465 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:33.469467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:33.470008 (+ 541us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:33.470208 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:33.470514 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 15:08:33.470826 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":121,"threads_started":1}
I20260811 15:08:33.474640 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:08:33.475651 3357 raft_consensus.cc:348] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df0a072fdbf345669f77895618e5f20a" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 40993 } }
I20260811 15:08:33.475907 3357 raft_consensus.cc:374] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:33.475968 3357 raft_consensus.cc:729] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df0a072fdbf345669f77895618e5f20a, State: Initialized, Role: FOLLOWER
I20260811 15:08:33.476397 3357 consensus_queue.cc:260] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df0a072fdbf345669f77895618e5f20a" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 40993 } }
I20260811 15:08:33.476532 3357 raft_consensus.cc:388] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:08:33.476612 3357 raft_consensus.cc:482] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:08:33.476717 3357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:33.477726 3357 raft_consensus.cc:504] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df0a072fdbf345669f77895618e5f20a" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 40993 } }
I20260811 15:08:33.478093 3357 leader_election.cc:302] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df0a072fdbf345669f77895618e5f20a; no voters:
I20260811 15:08:33.478428 3357 leader_election.cc:288] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:08:33.478525 3369 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:33.478711 3369 raft_consensus.cc:686] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a [term 1 LEADER]: Becoming Leader. State: Replica: df0a072fdbf345669f77895618e5f20a, State: Running, Role: LEADER
I20260811 15:08:33.479142 3369 consensus_queue.cc:237] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df0a072fdbf345669f77895618e5f20a" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 40993 } }
I20260811 15:08:33.479931 3357 sys_catalog.cc:564] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:08:33.480957 3370 sys_catalog.cc:455] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df0a072fdbf345669f77895618e5f20a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df0a072fdbf345669f77895618e5f20a" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 40993 } } }
I20260811 15:08:33.481098 3370 sys_catalog.cc:458] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:33.481419 3372 sys_catalog.cc:455] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a [sys.catalog]: SysCatalogTable state changed. Reason: New leader df0a072fdbf345669f77895618e5f20a. Latest consensus state: current_term: 1 leader_uuid: "df0a072fdbf345669f77895618e5f20a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df0a072fdbf345669f77895618e5f20a" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 40993 } } }
I20260811 15:08:33.481509 3372 sys_catalog.cc:458] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a [sys.catalog]: This master's current role is: LEADER
W20260811 15:08:33.485589 3392 catalog_manager.cc:1558] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:08:33.485697 3392 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:08:33.485780 3382 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:08:33.486550 3382 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:08:33.492554 3382 catalog_manager.cc:1347] Generated new cluster ID: 15d47e48562a4e0c8c0d8eb7f0617665
I20260811 15:08:33.492641 3382 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:08:33.499060 3361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:33.493075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50990 (local address 127.30.8.62:40993)
0811 15:08:33.493185 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:33.493189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:33.493202 (+ 13us) server_negotiation.cc:407] Connection header received
0811 15:08:33.493241 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:33.493243 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:33.493282 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:33.493353 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:33.494024 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:33.494841 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:33.496116 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:33.496478 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:33.496557 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:33.497799 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:33.497829 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:33.497833 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:08:33.498454 (+ 621us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:33.498498 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:08:33.498520 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:33.498588 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:33.498790 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 15:08:33.498870 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 15:08:33.505501 3382 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:08:33.506901 3382 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:08:33.520301 3382 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df0a072fdbf345669f77895618e5f20a: Generated new TSK 0
I20260811 15:08:33.521377 3382 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:08:33.532366 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 3252
2026-08-11T15:08:33Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1576 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:08:33 dist-test-slave-1vmg krb5kdc[3409](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:08:33 dist-test-slave-1vmg krb5kdc[3409](info): set up 2 sockets
Aug 11 15:08:33 dist-test-slave-1vmg krb5kdc[3409](info): commencing operation
krb5kdc: starting...
W20260811 15:08:34.697970 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:08:34 dist-test-slave-1vmg krb5kdc[3409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:08:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:08:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:08:34.827800 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:45913
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:45913
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:34.948932 3432 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:34.949288 3432 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:34.949358 3432 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:34.953554 3432 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:08:34.953661 3432 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:34.953696 3432 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:34.953722 3432 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:08:34.953748 3432 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:08:34.959465 3432 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:45913
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:45913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3432
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:34.960783 3432 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:34.961897 3432 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:34.970844 3437 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:34.970853 3438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:34.970798 3442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:34.971040 3432 server_base.cc:1034] running on GCE node
I20260811 15:08:34.971800 3432 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:34.973084 3432 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:34.974272 3432 hybrid_clock.cc:648] HybridClock initialized: now 1786460914974256 us; error 49 us; skew 500 ppm
Aug 11 15:08:34 dist-test-slave-1vmg krb5kdc[3409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460914, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:34.981112 3432 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:08:34.982223 3432 webserver.cc:460] Webserver started at http://127.30.8.62:45531/ using document root <none> and password file <none>
I20260811 15:08:34.982829 3432 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:34.982903 3432 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:34.983160 3432 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:34.985369 3432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "0a8121bb439e4d8ab7933488bc5ab3f5"
format_stamp: "Formatted at 2026-08-11 15:08:34 on dist-test-slave-1vmg"
server_key: "b2f66fa32a3f5ac507011d584441c75f"
server_key_iv: "bdae18d5da60017e4be8eec0d0f3343c"
server_key_version: "encryptionkey@0"
I20260811 15:08:34.985949 3432 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "0a8121bb439e4d8ab7933488bc5ab3f5"
format_stamp: "Formatted at 2026-08-11 15:08:34 on dist-test-slave-1vmg"
server_key: "b2f66fa32a3f5ac507011d584441c75f"
server_key_iv: "bdae18d5da60017e4be8eec0d0f3343c"
server_key_version: "encryptionkey@0"
I20260811 15:08:34.989863 3432 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.006s
I20260811 15:08:34.992419 3454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:34.993683 3432 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 15:08:34.993821 3432 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "0a8121bb439e4d8ab7933488bc5ab3f5"
format_stamp: "Formatted at 2026-08-11 15:08:34 on dist-test-slave-1vmg"
server_key: "b2f66fa32a3f5ac507011d584441c75f"
server_key_iv: "bdae18d5da60017e4be8eec0d0f3343c"
server_key_version: "encryptionkey@0"
I20260811 15:08:34.993930 3432 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:35.041687 3432 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:35.044735 3432 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:35.044937 3432 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:35.051947 3432 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:45913
I20260811 15:08:35.052224 3554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:45913 every 8 connection(s)
I20260811 15:08:35.052963 3432 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:08:35.055549 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 3432
I20260811 15:08:35.055651 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:08:35.055914 30752 external_mini_cluster.cc:1428] Setting key 98dc4589001570ef2d2b37726e6bed75
I20260811 15:08:35.056082 3556 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:35.063915 3556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5: Bootstrap starting.
Aug 11 15:08:35 dist-test-slave-1vmg krb5kdc[3409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:35.066542 3556 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:35.067559 3556 log.cc:824] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:35.070066 3556 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5: No bootstrap required, opened a new log
I20260811 15:08:35.071458 3561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:35.057370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59808 (local address 127.30.8.62:45913)
0811 15:08:35.057814 (+ 444us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:35.057836 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:35.057882 (+ 46us) server_negotiation.cc:407] Connection header received
0811 15:08:35.058963 (+ 1081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:35.058989 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:35.059438 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:35.059882 (+ 444us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:35.060529 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:35.062029 (+ 1500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:35.062879 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:35.063623 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:35.063803 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:35.066231 (+ 2428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:35.066238 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:35.066257 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:35.066297 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:35.068299 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:35.068725 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:35.068732 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:35.068739 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:35.068892 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:35.069124 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:35.069127 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:35.069128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:35.069691 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:35.069900 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:35.070086 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 15:08:35.070415 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":108,"threads_started":1}
I20260811 15:08:35.073760 3556 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a8121bb439e4d8ab7933488bc5ab3f5" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45913 } }
I20260811 15:08:35.074033 3556 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:35.074148 3556 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0a8121bb439e4d8ab7933488bc5ab3f5, State: Initialized, Role: FOLLOWER
I20260811 15:08:35.074611 3556 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a8121bb439e4d8ab7933488bc5ab3f5" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45913 } }
I20260811 15:08:35.074757 3556 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:08:35.074808 3556 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:08:35.074887 3556 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:35.076084 3556 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a8121bb439e4d8ab7933488bc5ab3f5" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45913 } }
I20260811 15:08:35.076417 3556 leader_election.cc:302] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0a8121bb439e4d8ab7933488bc5ab3f5; no voters:
I20260811 15:08:35.076722 3556 leader_election.cc:288] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:08:35.076839 3563 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:35.077060 3563 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5 [term 1 LEADER]: Becoming Leader. State: Replica: 0a8121bb439e4d8ab7933488bc5ab3f5, State: Running, Role: LEADER
I20260811 15:08:35.077432 3563 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a8121bb439e4d8ab7933488bc5ab3f5" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45913 } }
I20260811 15:08:35.077948 3556 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:08:35.079041 3568 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0a8121bb439e4d8ab7933488bc5ab3f5. Latest consensus state: current_term: 1 leader_uuid: "0a8121bb439e4d8ab7933488bc5ab3f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a8121bb439e4d8ab7933488bc5ab3f5" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45913 } } }
I20260811 15:08:35.079243 3568 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:35.079212 3567 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0a8121bb439e4d8ab7933488bc5ab3f5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0a8121bb439e4d8ab7933488bc5ab3f5" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45913 } } }
I20260811 15:08:35.079378 3567 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:35.079669 3583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:08:35.083552 3583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:08:35.089229 3583 catalog_manager.cc:1347] Generated new cluster ID: f1cfdb92752740dfbf03a50d90640aea
I20260811 15:08:35.089318 3583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:08:35.096364 3583 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:08:35.097323 3583 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:08:35.104108 3583 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0a8121bb439e4d8ab7933488bc5ab3f5: Generated new TSK 0
I20260811 15:08:35.104877 3583 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
I20260811 15:08:35.134465 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45913
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:35.252101 3598 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:35.252389 3598 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:35.252434 3598 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:35.256551 3598 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:35.256631 3598 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:35.256716 3598 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:08:35.262336 3598 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3598
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:35.263615 3598 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:35.264700 3598 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:35.274619 3609 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:35.274619 3612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:35.274648 3618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:35.274999 3598 server_base.cc:1034] running on GCE node
I20260811 15:08:35.275434 3598 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:35.276110 3598 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:35.277272 3598 hybrid_clock.cc:648] HybridClock initialized: now 1786460915277244 us; error 42 us; skew 500 ppm
Aug 11 15:08:35 dist-test-slave-1vmg krb5kdc[3409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460915, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:35.280184 3598 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:08:35.281132 3598 webserver.cc:460] Webserver started at http://127.30.8.1:43711/ using document root <none> and password file <none>
I20260811 15:08:35.281714 3598 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:35.281781 3598 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:35.281993 3598 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:35.284154 3598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "7e034fd6a69242d9a9159e31b0edfd4c"
format_stamp: "Formatted at 2026-08-11 15:08:35 on dist-test-slave-1vmg"
server_key: "b2ff28c09e37edbce37eceef7109ecb6"
server_key_iv: "6be17dead8dd13abb098b89fd0f0121a"
server_key_version: "encryptionkey@0"
I20260811 15:08:35.284720 3598 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "7e034fd6a69242d9a9159e31b0edfd4c"
format_stamp: "Formatted at 2026-08-11 15:08:35 on dist-test-slave-1vmg"
server_key: "b2ff28c09e37edbce37eceef7109ecb6"
server_key_iv: "6be17dead8dd13abb098b89fd0f0121a"
server_key_version: "encryptionkey@0"
I20260811 15:08:35.288684 3598 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:08:35.291219 3630 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:35.292505 3598 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 15:08:35.292624 3598 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "7e034fd6a69242d9a9159e31b0edfd4c"
format_stamp: "Formatted at 2026-08-11 15:08:35 on dist-test-slave-1vmg"
server_key: "b2ff28c09e37edbce37eceef7109ecb6"
server_key_iv: "6be17dead8dd13abb098b89fd0f0121a"
server_key_version: "encryptionkey@0"
I20260811 15:08:35.292735 3598 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:35.309682 3598 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:35.313174 3598 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:35.313388 3598 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:35.314021 3598 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:35.314899 3598 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:35.314992 3598 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:35.315047 3598 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:35.315075 3598 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:35.323977 3598 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:34273
I20260811 15:08:35.324004 3817 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:34273 every 8 connection(s)
I20260811 15:08:35.324931 3598 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:08:35.332383 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 3598
I20260811 15:08:35.332528 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:08:35.332831 30752 external_mini_cluster.cc:1428] Setting key 98d502eab41dc796c954e4c55b23c69c
Aug 11 15:08:35 dist-test-slave-1vmg krb5kdc[3409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460915, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.8.2@KRBTEST.COM; defaulting to no policy
I20260811 15:08:35.345124 3561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:35.329598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:40167 (local address 127.30.8.62:45913)
0811 15:08:35.329747 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:35.329751 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:35.331324 (+ 1573us) server_negotiation.cc:407] Connection header received
0811 15:08:35.333314 (+ 1990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:35.333318 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:35.333380 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:35.333433 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:35.335381 (+ 1948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:35.336096 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:35.337950 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:35.338252 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:35.338347 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:35.341221 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:35.341226 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:35.341228 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:35.341254 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:35.342581 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:35.343229 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:35.343232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:35.343234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:35.343325 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:35.343705 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:35.343708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:35.343710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:35.343936 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:35.344013 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:35.344799 (+ 786us) server_negotiation.cc:299] Negotiation successful
0811 15:08:35.344943 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 15:08:35.346153 3822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:35.330486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45913 (local address 127.30.8.1:40167)
0811 15:08:35.331129 (+ 643us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:35.331197 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:35.333026 (+ 1829us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:35.333906 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:35.333916 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:35.334442 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:35.335206 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:35.335229 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:35.336337 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:35.336340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:35.337828 (+ 1488us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:35.337839 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:35.338408 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:35.338412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:35.338606 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:35.339439 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:35.339458 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:35.341028 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:35.342703 (+ 1675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:35.342710 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:35.342729 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:35.343111 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:35.343421 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:35.343424 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:35.343426 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:35.343598 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:35.344080 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:35.344087 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:35.344453 (+ 366us) client_negotiation.cc:769] Sending connection context
0811 15:08:35.344669 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 15:08:35.345051 (+ 382us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":160,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.2@KRBTEST.COM" created.
I20260811 15:08:35.349889 3819 heartbeater.cc:347] Connected to a master server at 127.30.8.62:45913
I20260811 15:08:35.350232 3819 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:35.350831 3819 heartbeater.cc:511] Master 127.30.8.62:45913 requested a full tablet report, sending...
I20260811 15:08:35.352506 3486 ts_manager.cc:194] Registered new tserver with Master: 7e034fd6a69242d9a9159e31b0edfd4c (127.30.8.1:34273)
I20260811 15:08:35.353699 3486 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:40167
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
I20260811 15:08:35.372305 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45913
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:35.487845 3828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:35.488143 3828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:35.488200 3828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:35.492312 3828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:35.492391 3828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:35.492452 3828 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:08:35.498104 3828 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:35.499332 3828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:35.500367 3828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:35.510509 3836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:35.510504 3841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:35.510524 3835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:35.510906 3828 server_base.cc:1034] running on GCE node
I20260811 15:08:35.511441 3828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:35.512109 3828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:35.513260 3828 hybrid_clock.cc:648] HybridClock initialized: now 1786460915513233 us; error 42 us; skew 500 ppm
Aug 11 15:08:35 dist-test-slave-1vmg krb5kdc[3409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460915, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:35.516033 3828 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:08:35.516991 3828 webserver.cc:460] Webserver started at http://127.30.8.2:34173/ using document root <none> and password file <none>
I20260811 15:08:35.517560 3828 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:35.517621 3828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:35.517834 3828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:35.520020 3828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "5697929f33a84872863f9d9c825e7950"
format_stamp: "Formatted at 2026-08-11 15:08:35 on dist-test-slave-1vmg"
server_key: "2c28d4fd8c871de26dd1eae7e62ef801"
server_key_iv: "f2dff0a70dc69774b027cbfffc5e2f3e"
server_key_version: "encryptionkey@0"
I20260811 15:08:35.520581 3828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "5697929f33a84872863f9d9c825e7950"
format_stamp: "Formatted at 2026-08-11 15:08:35 on dist-test-slave-1vmg"
server_key: "2c28d4fd8c871de26dd1eae7e62ef801"
server_key_iv: "f2dff0a70dc69774b027cbfffc5e2f3e"
server_key_version: "encryptionkey@0"
I20260811 15:08:35.524482 3828 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:08:35.526906 3850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:35.528146 3828 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:08:35.528296 3828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "5697929f33a84872863f9d9c825e7950"
format_stamp: "Formatted at 2026-08-11 15:08:35 on dist-test-slave-1vmg"
server_key: "2c28d4fd8c871de26dd1eae7e62ef801"
server_key_iv: "f2dff0a70dc69774b027cbfffc5e2f3e"
server_key_version: "encryptionkey@0"
I20260811 15:08:35.528404 3828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:35.544471 3828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:35.547500 3828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:35.547696 3828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:35.548326 3828 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:35.549243 3828 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:35.549295 3828 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:35.549353 3828 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:35.549378 3828 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:35.558270 3828 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:35597
I20260811 15:08:35.558293 4038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:35597 every 8 connection(s)
I20260811 15:08:35.559299 3828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:08:35.559584 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 3828
I20260811 15:08:35.559717 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:08:35.560060 30752 external_mini_cluster.cc:1428] Setting key 0602fed7a6ad37c847fbc0cdcc04d22b
Aug 11 15:08:35 dist-test-slave-1vmg krb5kdc[3409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460915, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.8.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.3@KRBTEST.COM" created.
I20260811 15:08:35.574559 3561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:35.561266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:53527 (local address 127.30.8.62:45913)
0811 15:08:35.561403 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:35.561407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:35.562355 (+ 948us) server_negotiation.cc:407] Connection header received
0811 15:08:35.563884 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:35.563886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:35.563936 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:35.564007 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:35.565596 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:35.566127 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:35.567837 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:35.568067 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:35.568164 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:35.570832 (+ 2668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:35.570837 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:35.570839 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:35.570867 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:35.572159 (+ 1292us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:35.572743 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:35.572747 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:35.572749 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:35.572874 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:35.573241 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:35.573245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:35.573247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:35.573499 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:35.573604 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:35.574245 (+ 641us) server_negotiation.cc:299] Negotiation successful
0811 15:08:35.574389 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 15:08:35.575404 4046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:35.561688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45913 (local address 127.30.8.2:53527)
0811 15:08:35.562183 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:35.562239 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:35.563637 (+ 1398us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:35.564217 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:35.564226 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:35.564765 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:35.565452 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:35.565472 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:35.566264 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:35.566268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:35.567716 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:35.567729 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:35.568233 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:35.568238 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:35.568406 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:35.569025 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:35.569045 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:35.570597 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:35.572276 (+ 1679us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:35.572282 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:35.572300 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:35.572643 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:35.572963 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:35.572967 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:35.572969 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:35.573151 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:35.573630 (+ 479us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:35.573636 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:35.573989 (+ 353us) client_negotiation.cc:769] Sending connection context
0811 15:08:35.574168 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 15:08:35.574459 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":120,"threads_started":1}
I20260811 15:08:35.576555 4041 heartbeater.cc:347] Connected to a master server at 127.30.8.62:45913
I20260811 15:08:35.576829 4041 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:35.577427 4041 heartbeater.cc:511] Master 127.30.8.62:45913 requested a full tablet report, sending...
I20260811 15:08:35.578667 3486 ts_manager.cc:194] Registered new tserver with Master: 5697929f33a84872863f9d9c825e7950 (127.30.8.2:35597)
I20260811 15:08:35.579384 3486 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:53527
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
I20260811 15:08:35.589831 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45913
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:33137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:35.718026 4049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:35.718338 4049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:35.718390 4049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:35.722601 4049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:35.722693 4049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:35.722769 4049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:08:35.728384 4049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:33137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45913
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:35.729593 4049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:35.730657 4049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:35.740602 4060 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:35.740577 4056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:35.740574 4057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:35.740617 4049 server_base.cc:1034] running on GCE node
I20260811 15:08:35.741338 4049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:35.742054 4049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:35.743238 4049 hybrid_clock.cc:648] HybridClock initialized: now 1786460915743195 us; error 51 us; skew 500 ppm
Aug 11 15:08:35 dist-test-slave-1vmg krb5kdc[3409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460915, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:35.746078 4049 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:08:35.747148 4049 webserver.cc:460] Webserver started at http://127.30.8.3:34367/ using document root <none> and password file <none>
I20260811 15:08:35.747725 4049 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:35.747788 4049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:35.748005 4049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:35.750095 4049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "69fef05eea994b3ebfd5d510d9841841"
format_stamp: "Formatted at 2026-08-11 15:08:35 on dist-test-slave-1vmg"
server_key: "3fa1198931a903abb145362353efd085"
server_key_iv: "876c04a96f9fefea45829dc0c508b9e8"
server_key_version: "encryptionkey@0"
I20260811 15:08:35.750684 4049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "69fef05eea994b3ebfd5d510d9841841"
format_stamp: "Formatted at 2026-08-11 15:08:35 on dist-test-slave-1vmg"
server_key: "3fa1198931a903abb145362353efd085"
server_key_iv: "876c04a96f9fefea45829dc0c508b9e8"
server_key_version: "encryptionkey@0"
I20260811 15:08:35.754838 4049 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260811 15:08:35.757390 4072 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:35.758595 4049 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 15:08:35.758713 4049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "69fef05eea994b3ebfd5d510d9841841"
format_stamp: "Formatted at 2026-08-11 15:08:35 on dist-test-slave-1vmg"
server_key: "3fa1198931a903abb145362353efd085"
server_key_iv: "876c04a96f9fefea45829dc0c508b9e8"
server_key_version: "encryptionkey@0"
I20260811 15:08:35.758816 4049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:35.772300 4049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:35.775343 4049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:35.775534 4049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:35.776139 4049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:35.777021 4049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:35.777072 4049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:35.777127 4049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:35.777153 4049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:35.786435 4049 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:39299
I20260811 15:08:35.786481 4247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:39299 every 8 connection(s)
I20260811 15:08:35.787429 4049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:08:35.788719 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 4049
I20260811 15:08:35.788826 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:08:35.789129 30752 external_mini_cluster.cc:1428] Setting key 158b33a31b8329819b6f1c0979c5faaf
Aug 11 15:08:35 dist-test-slave-1vmg krb5kdc[3409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460915, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:35.801312 3561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:35.789362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:59201 (local address 127.30.8.62:45913)
0811 15:08:35.789500 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:35.789505 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:35.790451 (+ 946us) server_negotiation.cc:407] Connection header received
0811 15:08:35.792087 (+ 1636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:35.792091 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:35.792157 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:35.792229 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:35.793443 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:35.793973 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:35.795167 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:35.795390 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:35.795479 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:35.798064 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:35.798068 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:35.798071 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:35.798098 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:35.799472 (+ 1374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:35.799967 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:35.799969 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:35.799970 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:35.800045 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:35.800308 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:35.800310 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:35.800311 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:35.800492 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:35.800564 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:35.801065 (+ 501us) server_negotiation.cc:299] Negotiation successful
0811 15:08:35.801174 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 15:08:35.802274 4250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:35.789805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45913 (local address 127.30.8.3:59201)
0811 15:08:35.790298 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:35.790357 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:35.791874 (+ 1517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:35.792410 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:35.792417 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:35.792801 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:35.793298 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:35.793312 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:35.794096 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:35.794099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:35.795041 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:35.795052 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:35.795536 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:35.795540 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:35.795714 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:35.796288 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:35.796306 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:35.797891 (+ 1585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:35.799587 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:35.799593 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:35.799608 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:35.799889 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:35.800108 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:35.800110 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:35.800111 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:35.800243 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:35.800564 (+ 321us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:35.800569 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:35.800856 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 15:08:35.801025 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 15:08:35.801307 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":119,"threads_started":1}
I20260811 15:08:35.803408 4248 heartbeater.cc:347] Connected to a master server at 127.30.8.62:45913
I20260811 15:08:35.803665 4248 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:35.804169 4248 heartbeater.cc:511] Master 127.30.8.62:45913 requested a full tablet report, sending...
I20260811 15:08:35.805186 3485 ts_manager.cc:194] Registered new tserver with Master: 69fef05eea994b3ebfd5d510d9841841 (127.30.8.3:39299)
I20260811 15:08:35.805771 3485 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:59201
I20260811 15:08:35.815320 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 15:08:35 dist-test-slave-1vmg krb5kdc[3409](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460915, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 15:08:35 dist-test-slave-1vmg krb5kdc[3409](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460915, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:35.841506 3561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:35.833648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59818 (local address 127.30.8.62:45913)
0811 15:08:35.833817 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:35.833822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:35.833912 (+ 90us) server_negotiation.cc:407] Connection header received
0811 15:08:35.834040 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:35.834043 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:35.834088 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:35.834150 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:35.834825 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:35.835404 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:35.836194 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:35.836406 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:35.836473 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:35.838768 (+ 2295us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:35.838772 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:35.838774 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:35.838802 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:35.840124 (+ 1322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:35.840467 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:35.840470 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:35.840471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:35.840540 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:35.840726 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:35.840728 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:35.840729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:35.840955 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:35.841057 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:35.841232 (+ 175us) server_negotiation.cc:299] Negotiation successful
0811 15:08:35.841338 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 15:08:35.841944 3485 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:59818
I20260811 15:08:35.842878 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 3598
I20260811 15:08:35.856240 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 3828
I20260811 15:08:35.869547 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 4049
I20260811 15:08:35.882205 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 3432
2026-08-11T15:08:35Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2354 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:08:35 dist-test-slave-1vmg krb5kdc[4263](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:08:35 dist-test-slave-1vmg krb5kdc[4263](info): set up 2 sockets
Aug 11 15:08:35 dist-test-slave-1vmg krb5kdc[4263](info): commencing operation
krb5kdc: starting...
W20260811 15:08:37.064041 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:08:37 dist-test-slave-1vmg krb5kdc[4263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:08:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:08:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:08:37.190200 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:39801
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:33657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:39801
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:37.305371 4289 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:37.305673 4289 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:37.305728 4289 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:37.309828 4289 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:08:37.309917 4289 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:37.309945 4289 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:37.309965 4289 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:08:37.309983 4289 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:08:37.315553 4289 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:33657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:39801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:39801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4289
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:37.316743 4289 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:37.317800 4289 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:37.326398 4301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:37.326417 4299 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:37.326439 4295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:37.327152 4289 server_base.cc:1034] running on GCE node
I20260811 15:08:37.327884 4289 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:37.329439 4289 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:37.330614 4289 hybrid_clock.cc:648] HybridClock initialized: now 1786460917330550 us; error 84 us; skew 500 ppm
Aug 11 15:08:37 dist-test-slave-1vmg krb5kdc[4263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460917, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:37.339406 4289 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:08:37.340500 4289 webserver.cc:460] Webserver started at http://127.30.8.62:37959/ using document root <none> and password file <none>
I20260811 15:08:37.341109 4289 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:37.341178 4289 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:37.341478 4289 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:37.343726 4289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "0fe0d769828547a78d27f60e18c68f59"
format_stamp: "Formatted at 2026-08-11 15:08:37 on dist-test-slave-1vmg"
server_key: "fad3b89d2655f217f53663af1bd7f037"
server_key_iv: "bdfd6d8d86e53aae5e37ddd715084a72"
server_key_version: "encryptionkey@0"
I20260811 15:08:37.344290 4289 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "0fe0d769828547a78d27f60e18c68f59"
format_stamp: "Formatted at 2026-08-11 15:08:37 on dist-test-slave-1vmg"
server_key: "fad3b89d2655f217f53663af1bd7f037"
server_key_iv: "bdfd6d8d86e53aae5e37ddd715084a72"
server_key_version: "encryptionkey@0"
I20260811 15:08:37.348201 4289 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:08:37.350657 4312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:37.351975 4289 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:08:37.352101 4289 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "0fe0d769828547a78d27f60e18c68f59"
format_stamp: "Formatted at 2026-08-11 15:08:37 on dist-test-slave-1vmg"
server_key: "fad3b89d2655f217f53663af1bd7f037"
server_key_iv: "bdfd6d8d86e53aae5e37ddd715084a72"
server_key_version: "encryptionkey@0"
I20260811 15:08:37.352206 4289 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:37.383450 4289 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:37.386519 4289 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:37.386729 4289 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:37.393891 4289 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:39801
I20260811 15:08:37.393924 4391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:39801 every 8 connection(s)
I20260811 15:08:37.394901 4289 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:08:37.397774 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 4289
I20260811 15:08:37.397817 4394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:37.397878 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:08:37.398164 30752 external_mini_cluster.cc:1428] Setting key d0f992b70c7fd83ddf1c498531fdda1d
I20260811 15:08:37.404479 4394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59: Bootstrap starting.
I20260811 15:08:37.407002 4394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59: Neither blocks nor log segments found. Creating new log.
Aug 11 15:08:37 dist-test-slave-1vmg krb5kdc[4263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:37.407778 4394 log.cc:824] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:37.409898 4394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59: No bootstrap required, opened a new log
I20260811 15:08:37.412261 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:37.399620 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58126 (local address 127.30.8.62:39801)
0811 15:08:37.400120 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:37.400136 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:37.400175 (+ 39us) server_negotiation.cc:407] Connection header received
0811 15:08:37.401046 (+ 871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:37.401065 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:37.401416 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:37.401737 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:37.402549 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:37.403587 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:37.404395 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:37.404899 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:37.405050 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:37.407663 (+ 2613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:37.407669 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:37.407686 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:37.407715 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:37.409361 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:37.409862 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:37.409866 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:37.409872 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:37.409994 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:37.410294 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:37.410296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:37.410297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:37.410717 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:37.410869 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:37.411197 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 15:08:37.411497 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":135,"threads_started":1}
I20260811 15:08:37.413949 4394 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe0d769828547a78d27f60e18c68f59" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 39801 } }
I20260811 15:08:37.414243 4394 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:37.414305 4394 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fe0d769828547a78d27f60e18c68f59, State: Initialized, Role: FOLLOWER
I20260811 15:08:37.414770 4394 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe0d769828547a78d27f60e18c68f59" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 39801 } }
I20260811 15:08:37.414906 4394 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:08:37.414994 4394 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:08:37.415074 4394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:37.416020 4394 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe0d769828547a78d27f60e18c68f59" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 39801 } }
I20260811 15:08:37.416329 4394 leader_election.cc:302] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fe0d769828547a78d27f60e18c68f59; no voters:
I20260811 15:08:37.416596 4394 leader_election.cc:288] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:08:37.416733 4403 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:37.416939 4403 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59 [term 1 LEADER]: Becoming Leader. State: Replica: 0fe0d769828547a78d27f60e18c68f59, State: Running, Role: LEADER
I20260811 15:08:37.417263 4403 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe0d769828547a78d27f60e18c68f59" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 39801 } }
I20260811 15:08:37.417834 4394 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:08:37.418983 4407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fe0d769828547a78d27f60e18c68f59. Latest consensus state: current_term: 1 leader_uuid: "0fe0d769828547a78d27f60e18c68f59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe0d769828547a78d27f60e18c68f59" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 39801 } } }
I20260811 15:08:37.419116 4407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:37.419404 4404 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fe0d769828547a78d27f60e18c68f59" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fe0d769828547a78d27f60e18c68f59" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 39801 } } }
I20260811 15:08:37.419502 4404 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:37.419484 4419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:08:37.422365 4419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:08:37.428263 4419 catalog_manager.cc:1347] Generated new cluster ID: ed741def1d4f48948546e677aa940e4e
I20260811 15:08:37.428349 4419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:08:37.455282 4419 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:08:37.456657 4419 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:08:37.469736 4419 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fe0d769828547a78d27f60e18c68f59: Generated new TSK 0
I20260811 15:08:37.470504 4419 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
I20260811 15:08:37.511668 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:39801
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:33657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:37.628723 4435 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:37.629011 4435 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:37.629057 4435 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:37.633230 4435 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:37.633317 4435 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:37.633386 4435 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:08:37.639448 4435 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:33657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:39801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4435
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:37.640618 4435 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:37.641671 4435 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 15:08:37.651436 4435 server_base.cc:1034] running on GCE node
W20260811 15:08:37.651409 4443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:37.651386 4444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:37.651386 4446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:37.652010 4435 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:37.652657 4435 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:37.653816 4435 hybrid_clock.cc:648] HybridClock initialized: now 1786460917653800 us; error 33 us; skew 500 ppm
Aug 11 15:08:37 dist-test-slave-1vmg krb5kdc[4263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460917, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:37.656661 4435 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:08:37.657651 4435 webserver.cc:460] Webserver started at http://127.30.8.1:44945/ using document root <none> and password file <none>
I20260811 15:08:37.658234 4435 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:37.658303 4435 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:37.658526 4435 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:37.660708 4435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "b9334fe590114dfd97f0e54d9090f71e"
format_stamp: "Formatted at 2026-08-11 15:08:37 on dist-test-slave-1vmg"
server_key: "e751935af4916071bc0115d1d7924680"
server_key_iv: "99d9b36d68f6780baede6ba34a0458e4"
server_key_version: "encryptionkey@0"
I20260811 15:08:37.661273 4435 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "b9334fe590114dfd97f0e54d9090f71e"
format_stamp: "Formatted at 2026-08-11 15:08:37 on dist-test-slave-1vmg"
server_key: "e751935af4916071bc0115d1d7924680"
server_key_iv: "99d9b36d68f6780baede6ba34a0458e4"
server_key_version: "encryptionkey@0"
I20260811 15:08:37.665091 4435 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 15:08:37.667587 4458 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:37.668780 4435 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:08:37.668898 4435 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "b9334fe590114dfd97f0e54d9090f71e"
format_stamp: "Formatted at 2026-08-11 15:08:37 on dist-test-slave-1vmg"
server_key: "e751935af4916071bc0115d1d7924680"
server_key_iv: "99d9b36d68f6780baede6ba34a0458e4"
server_key_version: "encryptionkey@0"
I20260811 15:08:37.669000 4435 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:37.705677 4435 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:37.708695 4435 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:37.708899 4435 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:37.709528 4435 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:37.710404 4435 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:37.710453 4435 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:37.710508 4435 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:37.710537 4435 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:37.719466 4435 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:34059
I20260811 15:08:37.719506 4630 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:34059 every 8 connection(s)
I20260811 15:08:37.720438 4435 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
Aug 11 15:08:37 dist-test-slave-1vmg krb5kdc[4263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460917, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:37.730101 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 4435
I20260811 15:08:37.730216 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:08:37.730473 30752 external_mini_cluster.cc:1428] Setting key cd7bb970debb4a5b962b3ffbfdb86caa
I20260811 15:08:37.734177 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:37.722268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:58597 (local address 127.30.8.62:39801)
0811 15:08:37.722441 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:37.722446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:37.723389 (+ 943us) server_negotiation.cc:407] Connection header received
0811 15:08:37.724718 (+ 1329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:37.724721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:37.724770 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:37.724843 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:37.725990 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:37.726519 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:37.727649 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:37.727868 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:37.727941 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:37.730489 (+ 2548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:37.730492 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:37.730494 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:37.730523 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:37.731979 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:37.732560 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:37.732563 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:37.732564 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:37.732639 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:37.732986 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:37.732988 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:37.732989 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:37.733169 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:37.733239 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:37.733854 (+ 615us) server_negotiation.cc:299] Negotiation successful
0811 15:08:37.733986 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 15:08:37.735149 4634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:37.722684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:39801 (local address 127.30.8.1:58597)
0811 15:08:37.723213 (+ 529us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:37.723270 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:37.724529 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:37.725002 (+ 473us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:37.725008 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:37.725378 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:37.725876 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:37.725888 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:37.726609 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:37.726611 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:37.727549 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:37.727559 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:37.727984 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:37.727987 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:37.728135 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:37.728729 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:37.728746 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:37.730302 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:37.732091 (+ 1789us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:37.732098 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:37.732115 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:37.732474 (+ 359us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:37.732716 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:37.732719 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:37.732720 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:37.732907 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:37.733254 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:37.733261 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:37.733603 (+ 342us) client_negotiation.cc:769] Sending connection context
0811 15:08:37.733823 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 15:08:37.734199 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":109,"threads_started":1}
I20260811 15:08:37.736383 4631 heartbeater.cc:347] Connected to a master server at 127.30.8.62:39801
I20260811 15:08:37.736676 4631 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:37.737357 4631 heartbeater.cc:511] Master 127.30.8.62:39801 requested a full tablet report, sending...
I20260811 15:08:37.739096 4338 ts_manager.cc:194] Registered new tserver with Master: b9334fe590114dfd97f0e54d9090f71e (127.30.8.1:34059)
I20260811 15:08:37.740269 4338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:58597
WARNING: no policy specified for kudu/127.30.8.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
I20260811 15:08:37.757522 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:39801
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:33657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:37.873834 4641 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:37.874183 4641 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:37.874230 4641 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:37.878417 4641 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:37.878495 4641 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:37.878558 4641 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:08:37.884155 4641 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:33657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:39801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4641
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:37.885298 4641 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:37.886301 4641 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:37.896114 4647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:37.896136 4651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:37.896139 4648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:37.896380 4641 server_base.cc:1034] running on GCE node
I20260811 15:08:37.896979 4641 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:37.897647 4641 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:37.898808 4641 hybrid_clock.cc:648] HybridClock initialized: now 1786460917898778 us; error 42 us; skew 500 ppm
Aug 11 15:08:37 dist-test-slave-1vmg krb5kdc[4263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460917, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:37.901661 4641 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:08:37.902634 4641 webserver.cc:460] Webserver started at http://127.30.8.2:40977/ using document root <none> and password file <none>
I20260811 15:08:37.903270 4641 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:37.903339 4641 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:37.903558 4641 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:37.905746 4641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "a8125f3e30eb4701a9a1e6d88709dab8"
format_stamp: "Formatted at 2026-08-11 15:08:37 on dist-test-slave-1vmg"
server_key: "f325dad3463d3a3b6f684155e02467d3"
server_key_iv: "ca01902e141e325ac668bc006d91ed82"
server_key_version: "encryptionkey@0"
I20260811 15:08:37.906306 4641 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "a8125f3e30eb4701a9a1e6d88709dab8"
format_stamp: "Formatted at 2026-08-11 15:08:37 on dist-test-slave-1vmg"
server_key: "f325dad3463d3a3b6f684155e02467d3"
server_key_iv: "ca01902e141e325ac668bc006d91ed82"
server_key_version: "encryptionkey@0"
I20260811 15:08:37.910204 4641 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 15:08:37.913259 4664 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:37.914871 4641 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:08:37.915019 4641 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "a8125f3e30eb4701a9a1e6d88709dab8"
format_stamp: "Formatted at 2026-08-11 15:08:37 on dist-test-slave-1vmg"
server_key: "f325dad3463d3a3b6f684155e02467d3"
server_key_iv: "ca01902e141e325ac668bc006d91ed82"
server_key_version: "encryptionkey@0"
I20260811 15:08:37.915138 4641 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:37.931351 4641 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:37.934643 4641 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:37.934864 4641 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:37.935565 4641 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:37.936455 4641 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:37.936504 4641 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:37.936561 4641 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:37.936590 4641 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:37.945479 4641 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:45055
I20260811 15:08:37.945501 4854 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:45055 every 8 connection(s)
I20260811 15:08:37.946488 4641 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:08:37.955431 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 4641
I20260811 15:08:37.955541 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:08:37.955823 30752 external_mini_cluster.cc:1428] Setting key d90ff0f96c17101145426b7fca0e4df9
Aug 11 15:08:37 dist-test-slave-1vmg krb5kdc[4263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460917, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:37.960394 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:37.948380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:55195 (local address 127.30.8.62:39801)
0811 15:08:37.948518 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:37.948522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:37.949339 (+ 817us) server_negotiation.cc:407] Connection header received
0811 15:08:37.950718 (+ 1379us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:37.950721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:37.950781 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:37.950856 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:37.952108 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:37.952644 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:37.953776 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:37.953989 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:37.954080 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:37.956686 (+ 2606us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:37.956690 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:37.956693 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:37.956719 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:37.958268 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:37.958767 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:37.958771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:37.958773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:37.958869 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:37.959195 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:37.959197 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:37.959199 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:37.959436 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:37.959528 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:37.960165 (+ 637us) server_negotiation.cc:299] Negotiation successful
0811 15:08:37.960270 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 15:08:37.961447 4859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:37.948739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:39801 (local address 127.30.8.2:55195)
0811 15:08:37.949198 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:37.949239 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:37.950505 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:37.951074 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:37.951081 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:37.951452 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:37.951958 (+ 506us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:37.951971 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:37.952770 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:37.952773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:37.953647 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:37.953656 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:37.954134 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:37.954138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:37.954308 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:37.954865 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:37.954882 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:37.956518 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:37.958387 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:37.958393 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:37.958408 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:37.958667 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:37.958973 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:37.958976 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:37.958978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:37.959114 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:37.959539 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:37.959546 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:37.959915 (+ 369us) client_negotiation.cc:769] Sending connection context
0811 15:08:37.960136 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 15:08:37.960508 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":125,"threads_started":1}
I20260811 15:08:37.962615 4855 heartbeater.cc:347] Connected to a master server at 127.30.8.62:39801
I20260811 15:08:37.962863 4855 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:37.963390 4855 heartbeater.cc:511] Master 127.30.8.62:39801 requested a full tablet report, sending...
I20260811 15:08:37.964490 4338 ts_manager.cc:194] Registered new tserver with Master: a8125f3e30eb4701a9a1e6d88709dab8 (127.30.8.2:45055)
I20260811 15:08:37.965066 4338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:55195
WARNING: no policy specified for kudu/127.30.8.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
I20260811 15:08:37.983434 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:39801
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:33657
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:38.104959 4862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:38.105255 4862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:38.105304 4862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:38.109484 4862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:38.109573 4862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:38.109634 4862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:08:38.115236 4862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:33657
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:39801
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:38.116405 4862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:38.117470 4862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:38.127381 4870 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:38.127436 4868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:38.127379 4873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:38.127740 4862 server_base.cc:1034] running on GCE node
I20260811 15:08:38.128146 4862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:38.128791 4862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:38.129971 4862 hybrid_clock.cc:648] HybridClock initialized: now 1786460918129953 us; error 39 us; skew 500 ppm
Aug 11 15:08:38 dist-test-slave-1vmg krb5kdc[4263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:38.132717 4862 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:08:38.133646 4862 webserver.cc:460] Webserver started at http://127.30.8.3:45275/ using document root <none> and password file <none>
I20260811 15:08:38.134218 4862 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:38.134289 4862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:38.134505 4862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:38.136650 4862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "86c61f2a6bb54b059e2288a32c1b040b"
format_stamp: "Formatted at 2026-08-11 15:08:38 on dist-test-slave-1vmg"
server_key: "6dd2beb90aa608091a45b02a7001c81b"
server_key_iv: "16ed1f22515a875715f2e429b28d4d36"
server_key_version: "encryptionkey@0"
I20260811 15:08:38.137197 4862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "86c61f2a6bb54b059e2288a32c1b040b"
format_stamp: "Formatted at 2026-08-11 15:08:38 on dist-test-slave-1vmg"
server_key: "6dd2beb90aa608091a45b02a7001c81b"
server_key_iv: "16ed1f22515a875715f2e429b28d4d36"
server_key_version: "encryptionkey@0"
I20260811 15:08:38.141276 4862 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 15:08:38.143786 4880 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:38.145011 4862 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 15:08:38.145140 4862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "86c61f2a6bb54b059e2288a32c1b040b"
format_stamp: "Formatted at 2026-08-11 15:08:38 on dist-test-slave-1vmg"
server_key: "6dd2beb90aa608091a45b02a7001c81b"
server_key_iv: "16ed1f22515a875715f2e429b28d4d36"
server_key_version: "encryptionkey@0"
I20260811 15:08:38.145254 4862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:38.176725 4862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:38.179769 4862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:38.179976 4862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:38.180603 4862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:38.181470 4862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:38.181519 4862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:38.181576 4862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:38.181605 4862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:38.190493 4862 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:35885
I20260811 15:08:38.190521 5062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:35885 every 8 connection(s)
I20260811 15:08:38.191499 4862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
Aug 11 15:08:38 dist-test-slave-1vmg krb5kdc[4263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:38.201689 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 4862
I20260811 15:08:38.201817 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:08:38.202090 30752 external_mini_cluster.cc:1428] Setting key 47f89493208c2223306f9a005a2be231
I20260811 15:08:38.205188 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:38.193359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:49119 (local address 127.30.8.62:39801)
0811 15:08:38.193525 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:38.193530 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:38.194367 (+ 837us) server_negotiation.cc:407] Connection header received
0811 15:08:38.195774 (+ 1407us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:38.195777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:38.195826 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:38.195889 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:38.197052 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:38.197583 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:38.198733 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:38.199015 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:38.199104 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:38.201696 (+ 2592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:38.201699 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:38.201701 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:38.201728 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:38.203183 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:38.203688 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:38.203691 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:38.203692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:38.203769 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:38.204097 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:38.204100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:38.204102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:38.204293 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:38.204379 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:38.204904 (+ 525us) server_negotiation.cc:299] Negotiation successful
0811 15:08:38.205025 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 15:08:38.206036 5067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:38.193749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:39801 (local address 127.30.8.3:49119)
0811 15:08:38.194218 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:38.194262 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:38.195560 (+ 1298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:38.196045 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:38.196052 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:38.196437 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:38.196935 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:38.196949 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:38.197693 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:38.197696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:38.198599 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:38.198609 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:38.199144 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:38.199147 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:38.199305 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:38.199933 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:38.199952 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:38.201487 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:38.203297 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:38.203304 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:38.203322 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:38.203590 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:38.203853 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:38.203856 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:38.203858 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:38.204007 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:38.204392 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:38.204400 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:38.204661 (+ 261us) client_negotiation.cc:769] Sending connection context
0811 15:08:38.204846 (+ 185us) client_negotiation.cc:240] Negotiation successful
0811 15:08:38.205209 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":111,"threads_started":1}
I20260811 15:08:38.207109 5063 heartbeater.cc:347] Connected to a master server at 127.30.8.62:39801
I20260811 15:08:38.207363 5063 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:38.207878 5063 heartbeater.cc:511] Master 127.30.8.62:39801 requested a full tablet report, sending...
I20260811 15:08:38.208945 4338 ts_manager.cc:194] Registered new tserver with Master: 86c61f2a6bb54b059e2288a32c1b040b (127.30.8.3:35885)
I20260811 15:08:38.209518 4338 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:49119
I20260811 15:08:38.215853 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 15:08:38 dist-test-slave-1vmg krb5kdc[4263](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 15:08:38 dist-test-slave-1vmg krb5kdc[4263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.1@KRBTEST.COM
I20260811 15:08:38.244817 5076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:38.235909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59450 (local address 127.30.8.1:34059)
0811 15:08:38.236202 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:38.236209 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:38.236236 (+ 27us) server_negotiation.cc:407] Connection header received
0811 15:08:38.236315 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:38.236321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:38.236525 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:38.236651 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:38.237285 (+ 634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:38.238055 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:38.238811 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:38.239241 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:38.239325 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:38.241646 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:38.241657 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:38.241676 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:38.241704 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:38.243265 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:38.243627 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:38.243631 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:38.243633 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:38.243713 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:38.243896 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:38.243898 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:38.243899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:38.244153 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:38.244324 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:38.244486 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 15:08:38.244629 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":96,"threads_started":1}
Aug 11 15:08:38 dist-test-slave-1vmg krb5kdc[4263](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460918, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:38.255416 4399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:38.247697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58142 (local address 127.30.8.62:39801)
0811 15:08:38.247837 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:38.247841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:38.247856 (+ 15us) server_negotiation.cc:407] Connection header received
0811 15:08:38.247948 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:38.247951 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:38.247997 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:38.248061 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:38.248735 (+ 674us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:38.249330 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:38.250098 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:38.250331 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:38.250419 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:38.252726 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:38.252729 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:38.252731 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:38.252759 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:38.254055 (+ 1296us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:38.254377 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:38.254380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:38.254381 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:38.254452 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:38.254626 (+ 174us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:38.254628 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:38.254629 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:38.254865 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:38.254999 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:38.255163 (+ 164us) server_negotiation.cc:299] Negotiation successful
0811 15:08:38.255263 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 15:08:38.255806 4338 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58142
I20260811 15:08:38.256436 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 4435
I20260811 15:08:38.275555 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 4641
I20260811 15:08:38.288841 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 4862
I20260811 15:08:38.300801 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 4289
2026-08-11T15:08:38Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2414 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:08:38 dist-test-slave-1vmg krb5kdc[5082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:08:38 dist-test-slave-1vmg krb5kdc[5082](info): set up 2 sockets
Aug 11 15:08:38 dist-test-slave-1vmg krb5kdc[5082](info): commencing operation
krb5kdc: starting...
W20260811 15:08:39.481252 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:08:39 dist-test-slave-1vmg krb5kdc[5082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:08:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:08:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:08:39.607748 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:44883
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:44883
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:39.726629 5109 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:39.726996 5109 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:39.727074 5109 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:39.731274 5109 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:08:39.731372 5109 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:39.731406 5109 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:39.731432 5109 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:08:39.731460 5109 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:08:39.737103 5109 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:44883
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:44883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5109
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:39.738346 5109 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:39.739450 5109 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:39.748176 5117 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:39.748162 5120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:39.748162 5118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:39.748550 5109 server_base.cc:1034] running on GCE node
I20260811 15:08:39.749253 5109 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:39.750536 5109 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:39.751704 5109 hybrid_clock.cc:648] HybridClock initialized: now 1786460919751680 us; error 44 us; skew 500 ppm
Aug 11 15:08:39 dist-test-slave-1vmg krb5kdc[5082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460919, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:39.754640 5109 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:08:39.755506 5109 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:39.755584 5109 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:39.755818 5109 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:39.758061 5109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "1309710c884244a7adbf88df9ced4bfd"
format_stamp: "Formatted at 2026-08-11 15:08:39 on dist-test-slave-1vmg"
server_key: "e9aaa65d8387b107a2c8e820b85caa0b"
server_key_iv: "d1d64aa6820f8bf3ab8415784627f980"
server_key_version: "encryptionkey@0"
I20260811 15:08:39.758658 5109 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "1309710c884244a7adbf88df9ced4bfd"
format_stamp: "Formatted at 2026-08-11 15:08:39 on dist-test-slave-1vmg"
server_key: "e9aaa65d8387b107a2c8e820b85caa0b"
server_key_iv: "d1d64aa6820f8bf3ab8415784627f980"
server_key_version: "encryptionkey@0"
I20260811 15:08:39.762696 5109 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 15:08:39.765360 5131 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:39.766652 5109 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 15:08:39.766803 5109 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "1309710c884244a7adbf88df9ced4bfd"
format_stamp: "Formatted at 2026-08-11 15:08:39 on dist-test-slave-1vmg"
server_key: "e9aaa65d8387b107a2c8e820b85caa0b"
server_key_iv: "d1d64aa6820f8bf3ab8415784627f980"
server_key_version: "encryptionkey@0"
I20260811 15:08:39.766955 5109 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:39.797168 5109 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:39.800448 5109 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:39.800668 5109 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:39.808470 5109 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:44883
I20260811 15:08:39.808714 5216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:44883 every 8 connection(s)
I20260811 15:08:39.809219 5109 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:08:39.812433 5217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:39.815253 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 5109
I20260811 15:08:39.815359 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:08:39.815608 30752 external_mini_cluster.cc:1428] Setting key c3808c77a9ad9b2d88e2c20a92768021
I20260811 15:08:39.818746 5217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd: Bootstrap starting.
I20260811 15:08:39.821092 5217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:39.821856 5217 log.cc:824] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:39.824050 5217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd: No bootstrap required, opened a new log
Aug 11 15:08:39 dist-test-slave-1vmg krb5kdc[5082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:39.827180 5217 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1309710c884244a7adbf88df9ced4bfd" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } }
I20260811 15:08:39.827431 5217 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:39.827488 5217 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1309710c884244a7adbf88df9ced4bfd, State: Initialized, Role: FOLLOWER
I20260811 15:08:39.828018 5217 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1309710c884244a7adbf88df9ced4bfd" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } }
I20260811 15:08:39.828202 5217 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:08:39.828276 5217 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:08:39.828377 5217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:39.829646 5217 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1309710c884244a7adbf88df9ced4bfd" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } }
I20260811 15:08:39.830030 5217 leader_election.cc:302] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1309710c884244a7adbf88df9ced4bfd; no voters:
I20260811 15:08:39.830391 5217 leader_election.cc:288] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:08:39.830490 5220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:39.817099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48804 (local address 127.30.8.62:44883)
0811 15:08:39.817588 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:39.817603 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:39.817642 (+ 39us) server_negotiation.cc:407] Connection header received
0811 15:08:39.818281 (+ 639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:39.818300 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:39.818637 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:39.818984 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:39.819772 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:39.820767 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:39.821637 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:39.822141 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:39.822302 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:39.824860 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:39.824866 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:39.824882 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:39.824916 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:39.827199 (+ 2283us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:39.827654 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:39.827661 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:39.827667 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:39.827818 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:39.828079 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:39.828083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:39.828085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:39.828661 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:39.828872 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:39.829092 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 15:08:39.829426 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":149,"threads_started":1}
I20260811 15:08:39.830515 5223 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:39.830730 5223 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd [term 1 LEADER]: Becoming Leader. State: Replica: 1309710c884244a7adbf88df9ced4bfd, State: Running, Role: LEADER
I20260811 15:08:39.831113 5223 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1309710c884244a7adbf88df9ced4bfd" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } }
I20260811 15:08:39.831758 5217 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:08:39.833455 5226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1309710c884244a7adbf88df9ced4bfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1309710c884244a7adbf88df9ced4bfd" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } } }
I20260811 15:08:39.833577 5226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:39.833953 5227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1309710c884244a7adbf88df9ced4bfd. Latest consensus state: current_term: 1 leader_uuid: "1309710c884244a7adbf88df9ced4bfd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1309710c884244a7adbf88df9ced4bfd" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } } }
I20260811 15:08:39.834084 5227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:39.834270 5243 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:08:39.837495 5243 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:08:39.843461 5243 catalog_manager.cc:1347] Generated new cluster ID: cf6ebc7ee45b4aea952f18a8667af42e
I20260811 15:08:39.843549 5243 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:08:39.851742 5243 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:08:39.852937 5243 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:08:39.857928 5243 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd: Generated new TSK 0
I20260811 15:08:39.858810 5243 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
I20260811 15:08:39.891103 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:44883
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:40.010720 5256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:40.011088 5256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:40.011168 5256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:40.015290 5256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:40.015373 5256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:40.015461 5256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:08:40.021086 5256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:44883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:40.022306 5256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:40.023411 5256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:40.033084 5266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:40.033066 5263 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:40.033250 5256 server_base.cc:1034] running on GCE node
W20260811 15:08:40.033267 5264 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:40.033854 5256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:40.034533 5256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:40.035692 5256 hybrid_clock.cc:648] HybridClock initialized: now 1786460920035673 us; error 37 us; skew 500 ppm
Aug 11 15:08:40 dist-test-slave-1vmg krb5kdc[5082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460920, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:40.042775 5256 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:08:40.043660 5256 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:40.043737 5256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:40.043970 5256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:40.046139 5256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "e0dafc7f5dcf4896840c7194b6ca46d6"
format_stamp: "Formatted at 2026-08-11 15:08:40 on dist-test-slave-1vmg"
server_key: "fe80176babd3fe782a57e6e7a0bd3bc7"
server_key_iv: "7931d02b790d431da944f9e36b6080bd"
server_key_version: "encryptionkey@0"
I20260811 15:08:40.046715 5256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "e0dafc7f5dcf4896840c7194b6ca46d6"
format_stamp: "Formatted at 2026-08-11 15:08:40 on dist-test-slave-1vmg"
server_key: "fe80176babd3fe782a57e6e7a0bd3bc7"
server_key_iv: "7931d02b790d431da944f9e36b6080bd"
server_key_version: "encryptionkey@0"
I20260811 15:08:40.050622 5256 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 15:08:40.053303 5276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:40.054562 5256 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 15:08:40.054685 5256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "e0dafc7f5dcf4896840c7194b6ca46d6"
format_stamp: "Formatted at 2026-08-11 15:08:40 on dist-test-slave-1vmg"
server_key: "fe80176babd3fe782a57e6e7a0bd3bc7"
server_key_iv: "7931d02b790d431da944f9e36b6080bd"
server_key_version: "encryptionkey@0"
I20260811 15:08:40.054795 5256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:40.069527 5256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:40.072974 5256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:40.073164 5256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:40.073797 5256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:40.074687 5256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:40.074736 5256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:40.074782 5256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:40.074811 5256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:40.083446 5256 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:34989
I20260811 15:08:40.083467 5470 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:34989 every 8 connection(s)
I20260811 15:08:40.084167 5256 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:08:40.089123 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 5256
I20260811 15:08:40.089262 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:08:40.089560 30752 external_mini_cluster.cc:1428] Setting key d4aa3d4181f9d452007dcccd8a9711ed
Aug 11 15:08:40 dist-test-slave-1vmg krb5kdc[5082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460920, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:40.098428 5220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:40.085998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:43641 (local address 127.30.8.62:44883)
0811 15:08:40.086182 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:40.086187 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:40.087049 (+ 862us) server_negotiation.cc:407] Connection header received
0811 15:08:40.088355 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:40.088358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:40.088407 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:40.088481 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:40.090158 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.090703 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.091916 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.092148 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.092239 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:40.094893 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:40.094896 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:40.094899 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:40.094954 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:40.096355 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:40.096871 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:40.096874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:40.096875 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:40.096953 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:40.097278 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:40.097282 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:40.097284 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:40.097475 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:40.097563 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:40.098111 (+ 548us) server_negotiation.cc:299] Negotiation successful
0811 15:08:40.098254 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
WARNING: no policy specified for kudu/127.30.8.2@KRBTEST.COM; defaulting to no policy
I20260811 15:08:40.099355 5477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:40.086410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:44883 (local address 127.30.8.1:43641)
0811 15:08:40.086876 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:40.086950 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:40.088147 (+ 1197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:40.088633 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:40.088640 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:40.089113 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:40.089826 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:40.089847 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.090838 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.090841 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:40.091790 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:40.091800 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.092301 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.092306 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:40.092467 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:40.093043 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:40.093061 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:40.094714 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:40.096472 (+ 1758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:40.096479 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:40.096498 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:40.096773 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:40.097037 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:40.097040 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:40.097042 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:40.097189 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:40.097585 (+ 396us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:40.097592 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:40.097863 (+ 271us) client_negotiation.cc:769] Sending connection context
0811 15:08:40.098044 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 15:08:40.098407 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":124,"threads_started":1}
I20260811 15:08:40.100651 5471 heartbeater.cc:347] Connected to a master server at 127.30.8.62:44883
I20260811 15:08:40.100896 5471 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:40.101424 5471 heartbeater.cc:511] Master 127.30.8.62:44883 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.2@KRBTEST.COM" created.
I20260811 15:08:40.103255 5155 ts_manager.cc:194] Registered new tserver with Master: e0dafc7f5dcf4896840c7194b6ca46d6 (127.30.8.1:34989)
I20260811 15:08:40.104596 5155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:43641
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
I20260811 15:08:40.117631 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:44883
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:40.234015 5481 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:40.234314 5481 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:40.234360 5481 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:40.238425 5481 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:40.238508 5481 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:40.238576 5481 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:08:40.244074 5481 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:44883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5481
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:40.245221 5481 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:40.246241 5481 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:40.255924 5493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:40.255910 5494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:40.255967 5481 server_base.cc:1034] running on GCE node
W20260811 15:08:40.255931 5497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:40.256664 5481 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:40.257375 5481 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:40.258538 5481 hybrid_clock.cc:648] HybridClock initialized: now 1786460920258517 us; error 42 us; skew 500 ppm
Aug 11 15:08:40 dist-test-slave-1vmg krb5kdc[5082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460920, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:40.261372 5481 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:08:40.262327 5481 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:40.262406 5481 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:40.262621 5481 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:40.264801 5481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "1b43bc7c26bf43d4a7671199c695892c"
format_stamp: "Formatted at 2026-08-11 15:08:40 on dist-test-slave-1vmg"
server_key: "347f98c76bc370e11a30c0487a653b9d"
server_key_iv: "afb82b3e64a24e8ae1c2684a1e5d8bb2"
server_key_version: "encryptionkey@0"
I20260811 15:08:40.265381 5481 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "1b43bc7c26bf43d4a7671199c695892c"
format_stamp: "Formatted at 2026-08-11 15:08:40 on dist-test-slave-1vmg"
server_key: "347f98c76bc370e11a30c0487a653b9d"
server_key_iv: "afb82b3e64a24e8ae1c2684a1e5d8bb2"
server_key_version: "encryptionkey@0"
I20260811 15:08:40.269219 5481 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:08:40.271844 5509 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:40.273200 5481 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:08:40.273316 5481 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "1b43bc7c26bf43d4a7671199c695892c"
format_stamp: "Formatted at 2026-08-11 15:08:40 on dist-test-slave-1vmg"
server_key: "347f98c76bc370e11a30c0487a653b9d"
server_key_iv: "afb82b3e64a24e8ae1c2684a1e5d8bb2"
server_key_version: "encryptionkey@0"
I20260811 15:08:40.273418 5481 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:40.297194 5481 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:40.300596 5481 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:40.300791 5481 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:40.301393 5481 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:40.302276 5481 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:40.302325 5481 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:40.302371 5481 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:40.302398 5481 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:40.311313 5481 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:45819
I20260811 15:08:40.311328 5688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:45819 every 8 connection(s)
I20260811 15:08:40.311971 5481 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:08:40.315308 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 5481
I20260811 15:08:40.315428 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:08:40.315701 30752 external_mini_cluster.cc:1428] Setting key 1e55b2ed41e95acb301aea62504f11b7
Aug 11 15:08:40 dist-test-slave-1vmg krb5kdc[5082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460920, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.8.3@KRBTEST.COM; defaulting to no policy
I20260811 15:08:40.327379 5220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:40.313760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:56921 (local address 127.30.8.62:44883)
0811 15:08:40.313938 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:40.313942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:40.314823 (+ 881us) server_negotiation.cc:407] Connection header received
0811 15:08:40.316297 (+ 1474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:40.316301 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:40.316357 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:40.316440 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:40.318187 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.318903 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.320593 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.320878 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.320976 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:40.323564 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:40.323586 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:40.323589 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:40.323620 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:40.325200 (+ 1580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:40.325729 (+ 529us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:40.325733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:40.325735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:40.325818 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:40.326138 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:40.326142 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:40.326143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:40.326346 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:40.326436 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:40.327071 (+ 635us) server_negotiation.cc:299] Negotiation successful
0811 15:08:40.327195 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 15:08:40.328249 5693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:40.314171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:44883 (local address 127.30.8.2:56921)
0811 15:08:40.314662 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:40.314706 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:40.316024 (+ 1318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:40.317096 (+ 1072us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:40.317105 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:40.317496 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:40.318028 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:40.318042 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.319075 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.319078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:40.320468 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:40.320481 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.321038 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.321040 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:40.321201 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:40.321805 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:40.321824 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:40.323310 (+ 1486us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:40.325323 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:40.325330 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:40.325350 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:40.325621 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:40.325900 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:40.325903 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:40.325905 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:40.326052 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:40.326490 (+ 438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:40.326497 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:40.326789 (+ 292us) client_negotiation.cc:769] Sending connection context
0811 15:08:40.327007 (+ 218us) client_negotiation.cc:240] Negotiation successful
0811 15:08:40.327351 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":125,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.3@KRBTEST.COM" created.
I20260811 15:08:40.329560 5689 heartbeater.cc:347] Connected to a master server at 127.30.8.62:44883
I20260811 15:08:40.329819 5689 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:40.330317 5689 heartbeater.cc:511] Master 127.30.8.62:44883 requested a full tablet report, sending...
I20260811 15:08:40.331525 5155 ts_manager.cc:194] Registered new tserver with Master: 1b43bc7c26bf43d4a7671199c695892c (127.30.8.2:45819)
I20260811 15:08:40.332191 5155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:56921
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
I20260811 15:08:40.345734 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:44883
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:43839
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:40.463409 5697 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:40.463726 5697 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:40.463795 5697 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:40.467947 5697 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:40.468034 5697 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:40.468117 5697 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:08:40.473785 5697 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:43839
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:44883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5697
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:40.475005 5697 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:40.476039 5697 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 15:08:40.485942 5697 server_base.cc:1034] running on GCE node
W20260811 15:08:40.485888 5711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:40.485888 5709 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:40.485899 5713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:40.486594 5697 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:40.487325 5697 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:40.488488 5697 hybrid_clock.cc:648] HybridClock initialized: now 1786460920488465 us; error 36 us; skew 500 ppm
Aug 11 15:08:40 dist-test-slave-1vmg krb5kdc[5082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460920, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:40.491267 5697 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:08:40.492054 5697 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:40.492126 5697 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:40.492332 5697 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:40.494511 5697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "c531986793ea429bbd0c9382b0232dd1"
format_stamp: "Formatted at 2026-08-11 15:08:40 on dist-test-slave-1vmg"
server_key: "8542261669064736dbdf8f2c95e4563f"
server_key_iv: "13ee9ad8aee4df9e413c9c8c9404f167"
server_key_version: "encryptionkey@0"
I20260811 15:08:40.495113 5697 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "c531986793ea429bbd0c9382b0232dd1"
format_stamp: "Formatted at 2026-08-11 15:08:40 on dist-test-slave-1vmg"
server_key: "8542261669064736dbdf8f2c95e4563f"
server_key_iv: "13ee9ad8aee4df9e413c9c8c9404f167"
server_key_version: "encryptionkey@0"
I20260811 15:08:40.499125 5697 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 15:08:40.501672 5721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:40.502966 5697 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:08:40.503098 5697 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "c531986793ea429bbd0c9382b0232dd1"
format_stamp: "Formatted at 2026-08-11 15:08:40 on dist-test-slave-1vmg"
server_key: "8542261669064736dbdf8f2c95e4563f"
server_key_iv: "13ee9ad8aee4df9e413c9c8c9404f167"
server_key_version: "encryptionkey@0"
I20260811 15:08:40.503208 5697 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:40.516471 5697 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:40.519804 5697 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:40.520004 5697 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:40.520610 5697 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:40.521500 5697 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:40.521550 5697 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:40.521605 5697 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:40.521633 5697 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:40.530539 5697 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:34593
I20260811 15:08:40.530573 5909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:34593 every 8 connection(s)
I20260811 15:08:40.531195 5697 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:08:40.533522 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 5697
I20260811 15:08:40.533638 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:08:40.533957 30752 external_mini_cluster.cc:1428] Setting key af680c3c432c6d1cf1f5a506bfce7c15
Aug 11 15:08:40 dist-test-slave-1vmg krb5kdc[5082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460920, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:40.544335 5220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:40.532982 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:60615 (local address 127.30.8.62:44883)
0811 15:08:40.533157 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:40.533162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:40.534122 (+ 960us) server_negotiation.cc:407] Connection header received
0811 15:08:40.535685 (+ 1563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:40.535688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:40.535744 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:40.535821 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:40.537035 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.537569 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.538647 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.538850 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.538943 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:40.541321 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:40.541324 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:40.541326 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:40.541351 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:40.542496 (+ 1145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:40.543003 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:40.543006 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:40.543007 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:40.543078 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:40.543356 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:40.543358 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:40.543360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:40.543541 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:40.543613 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:40.544090 (+ 477us) server_negotiation.cc:299] Negotiation successful
0811 15:08:40.544205 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 15:08:40.545185 5912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:40.533456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:44883 (local address 127.30.8.3:60615)
0811 15:08:40.533965 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:40.534020 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:40.535479 (+ 1459us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:40.536024 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:40.536031 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:40.536403 (+ 372us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:40.536911 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:40.536924 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.537661 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.537663 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:40.538558 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:40.538568 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.538979 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.538982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:40.539128 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:40.539661 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:40.539677 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:40.541150 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:40.542597 (+ 1447us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:40.542603 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:40.542617 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:40.542885 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:40.543145 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:40.543147 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:40.543148 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:40.543286 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:40.543612 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:40.543617 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:40.543886 (+ 269us) client_negotiation.cc:769] Sending connection context
0811 15:08:40.544046 (+ 160us) client_negotiation.cc:240] Negotiation successful
0811 15:08:40.544318 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":128,"threads_started":1}
I20260811 15:08:40.546358 5910 heartbeater.cc:347] Connected to a master server at 127.30.8.62:44883
I20260811 15:08:40.546626 5910 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:40.547199 5910 heartbeater.cc:511] Master 127.30.8.62:44883 requested a full tablet report, sending...
I20260811 15:08:40.548264 5155 ts_manager.cc:194] Registered new tserver with Master: c531986793ea429bbd0c9382b0232dd1 (127.30.8.3:34593)
I20260811 15:08:40.548835 5155 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:60615
I20260811 15:08:40.559764 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 15:08:40.568562 5220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:40.561699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48814 (local address 127.30.8.62:44883)
0811 15:08:40.561826 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:40.561831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:40.561991 (+ 160us) server_negotiation.cc:407] Connection header received
0811 15:08:40.562116 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:40.562119 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:40.562167 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:40.562244 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:40.562993 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.563512 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.564308 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.564518 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.564605 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:40.565894 (+ 1289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:40.565898 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:40.565900 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:40.565925 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:40.566989 (+ 1064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:40.567374 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:40.567377 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:40.567379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:40.567456 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:40.567696 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:40.567699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:40.567701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:40.567936 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:40.568057 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:40.568284 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 15:08:40.568400 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 15:08:40.571436 5155 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48814:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:08:40.574451 5155 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:08:40.589036 5935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:40.583233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:34593 (local address 127.0.0.1:52614)
0811 15:08:40.584274 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:40.584286 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:40.584368 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:40.585076 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:40.585079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:40.585093 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:40.585225 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:40.585232 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.586237 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.586241 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:40.587450 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:40.587459 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.588447 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.588450 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:40.588570 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:40.588609 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 15:08:40.588740 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 15:08:40.588872 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":984,"mutex_wait_us":26,"thread_start_us":625,"threads_started":1}
I20260811 15:08:40.589267 5936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:40.583548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52614 (local address 127.30.8.3:34593)
0811 15:08:40.584138 (+ 590us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:40.584156 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:40.584513 (+ 357us) server_negotiation.cc:407] Connection header received
0811 15:08:40.584613 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:40.584617 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:40.584794 (+ 177us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:40.584915 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:40.585356 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.586107 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.587564 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.588124 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.588222 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:40.588381 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:40.588807 (+ 426us) server_negotiation.cc:299] Negotiation successful
0811 15:08:40.589014 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":495,"thread_start_us":126,"threads_started":1}
I20260811 15:08:40.589634 5930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:40.583160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42202 (local address 127.30.8.2:45819)
0811 15:08:40.583669 (+ 509us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:40.583687 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:40.584137 (+ 450us) server_negotiation.cc:407] Connection header received
0811 15:08:40.584393 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:40.584399 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:40.584598 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:40.584739 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:40.585313 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.586531 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.588256 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.588818 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.588918 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:40.589050 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:40.589384 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 15:08:40.589489 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":153,"threads_started":1}
I20260811 15:08:40.590054 5939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:40.583140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:45819 (local address 127.0.0.1:42202)
0811 15:08:40.584037 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:40.584050 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:40.584211 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:40.584966 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:40.584971 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:40.585005 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:40.585152 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:40.585159 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.586664 (+ 1505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.586667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:40.588090 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:40.588102 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.589116 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.589120 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:40.589214 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:40.589238 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 15:08:40.589845 (+ 607us) client_negotiation.cc:240] Negotiation successful
0811 15:08:40.589927 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":838,"mutex_wait_us":157,"spinlock_wait_cycles":5376,"thread_start_us":81,"threads_started":1}
I20260811 15:08:40.590842 5928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:40.583046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32844 (local address 127.30.8.1:34989)
0811 15:08:40.583899 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:40.583905 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:40.584058 (+ 153us) server_negotiation.cc:407] Connection header received
0811 15:08:40.584381 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:40.584388 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:40.584602 (+ 214us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:40.584729 (+ 127us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:40.585388 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.586500 (+ 1112us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.588230 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.590033 (+ 1803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.590129 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:40.590280 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:40.590586 (+ 306us) server_negotiation.cc:299] Negotiation successful
0811 15:08:40.590687 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"thread_start_us":121,"threads_started":1}
I20260811 15:08:40.591305 5929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:40.582884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:34989 (local address 127.0.0.1:32844)
0811 15:08:40.583913 (+ 1029us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:40.583967 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:40.584177 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:40.584968 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:40.584971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:40.585006 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:40.585230 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:40.585237 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.586664 (+ 1427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.586667 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:40.588090 (+ 1423us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:40.588102 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.590342 (+ 2240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.590345 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:40.590429 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:40.590451 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 15:08:40.591085 (+ 634us) client_negotiation.cc:240] Negotiation successful
0811 15:08:40.591167 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"mutex_wait_us":114,"spinlock_wait_cycles":3712,"thread_start_us":101,"threads_started":1}
I20260811 15:08:40.592778 5590 tablet_service.cc:1467] Processing CreateTablet for tablet dc2378a174cb49c1bbbe62aee7d84f04 (DEFAULT_TABLE table=test-table [id=3a8a2add9c0f42ea9e8ed6620aa8d38b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:40.592799 5801 tablet_service.cc:1467] Processing CreateTablet for tablet dc2378a174cb49c1bbbe62aee7d84f04 (DEFAULT_TABLE table=test-table [id=3a8a2add9c0f42ea9e8ed6620aa8d38b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:40.593802 5590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc2378a174cb49c1bbbe62aee7d84f04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:40.593858 5801 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc2378a174cb49c1bbbe62aee7d84f04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:40.593990 5363 tablet_service.cc:1467] Processing CreateTablet for tablet dc2378a174cb49c1bbbe62aee7d84f04 (DEFAULT_TABLE table=test-table [id=3a8a2add9c0f42ea9e8ed6620aa8d38b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:40.595060 5363 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc2378a174cb49c1bbbe62aee7d84f04. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:40.601899 5943 tablet_bootstrap.cc:492] T dc2378a174cb49c1bbbe62aee7d84f04 P e0dafc7f5dcf4896840c7194b6ca46d6: Bootstrap starting.
I20260811 15:08:40.602337 5940 tablet_bootstrap.cc:492] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c: Bootstrap starting.
I20260811 15:08:40.603286 5942 tablet_bootstrap.cc:492] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1: Bootstrap starting.
I20260811 15:08:40.603961 5943 tablet_bootstrap.cc:654] T dc2378a174cb49c1bbbe62aee7d84f04 P e0dafc7f5dcf4896840c7194b6ca46d6: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:40.604290 5940 tablet_bootstrap.cc:654] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:40.604722 5943 log.cc:824] T dc2378a174cb49c1bbbe62aee7d84f04 P e0dafc7f5dcf4896840c7194b6ca46d6: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:40.605298 5942 tablet_bootstrap.cc:654] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:40.605544 5940 log.cc:824] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:40.606506 5942 log.cc:824] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:40.606813 5943 tablet_bootstrap.cc:492] T dc2378a174cb49c1bbbe62aee7d84f04 P e0dafc7f5dcf4896840c7194b6ca46d6: No bootstrap required, opened a new log
I20260811 15:08:40.607053 5943 ts_tablet_manager.cc:1397] T dc2378a174cb49c1bbbe62aee7d84f04 P e0dafc7f5dcf4896840c7194b6ca46d6: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 15:08:40.608296 5942 tablet_bootstrap.cc:492] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1: No bootstrap required, opened a new log
I20260811 15:08:40.608448 5942 ts_tablet_manager.cc:1397] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 15:08:40.610457 5943 raft_consensus.cc:348] T dc2378a174cb49c1bbbe62aee7d84f04 P e0dafc7f5dcf4896840c7194b6ca46d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0dafc7f5dcf4896840c7194b6ca46d6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 34989 } } peers { permanent_uuid: "1b43bc7c26bf43d4a7671199c695892c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 45819 } } peers { permanent_uuid: "c531986793ea429bbd0c9382b0232dd1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 34593 } }
I20260811 15:08:40.610698 5943 raft_consensus.cc:374] T dc2378a174cb49c1bbbe62aee7d84f04 P e0dafc7f5dcf4896840c7194b6ca46d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:40.610757 5943 raft_consensus.cc:729] T dc2378a174cb49c1bbbe62aee7d84f04 P e0dafc7f5dcf4896840c7194b6ca46d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0dafc7f5dcf4896840c7194b6ca46d6, State: Initialized, Role: FOLLOWER
I20260811 15:08:40.611260 5943 consensus_queue.cc:260] T dc2378a174cb49c1bbbe62aee7d84f04 P e0dafc7f5dcf4896840c7194b6ca46d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0dafc7f5dcf4896840c7194b6ca46d6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 34989 } } peers { permanent_uuid: "1b43bc7c26bf43d4a7671199c695892c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 45819 } } peers { permanent_uuid: "c531986793ea429bbd0c9382b0232dd1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 34593 } }
I20260811 15:08:40.611155 5942 raft_consensus.cc:348] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0dafc7f5dcf4896840c7194b6ca46d6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 34989 } } peers { permanent_uuid: "1b43bc7c26bf43d4a7671199c695892c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 45819 } } peers { permanent_uuid: "c531986793ea429bbd0c9382b0232dd1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 34593 } }
I20260811 15:08:40.611404 5942 raft_consensus.cc:374] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:40.611469 5942 raft_consensus.cc:729] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c531986793ea429bbd0c9382b0232dd1, State: Initialized, Role: FOLLOWER
I20260811 15:08:40.611927 5471 heartbeater.cc:503] Master 127.30.8.62:44883 was elected leader, sending a full tablet report...
I20260811 15:08:40.611883 5942 consensus_queue.cc:260] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0dafc7f5dcf4896840c7194b6ca46d6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 34989 } } peers { permanent_uuid: "1b43bc7c26bf43d4a7671199c695892c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 45819 } } peers { permanent_uuid: "c531986793ea429bbd0c9382b0232dd1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 34593 } }
I20260811 15:08:40.612285 5943 ts_tablet_manager.cc:1428] T dc2378a174cb49c1bbbe62aee7d84f04 P e0dafc7f5dcf4896840c7194b6ca46d6: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 15:08:40.612424 5940 tablet_bootstrap.cc:492] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c: No bootstrap required, opened a new log
I20260811 15:08:40.612627 5940 ts_tablet_manager.cc:1397] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260811 15:08:40.612766 5942 ts_tablet_manager.cc:1428] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260811 15:08:40.613075 5910 heartbeater.cc:503] Master 127.30.8.62:44883 was elected leader, sending a full tablet report...
I20260811 15:08:40.616101 5940 raft_consensus.cc:348] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0dafc7f5dcf4896840c7194b6ca46d6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 34989 } } peers { permanent_uuid: "1b43bc7c26bf43d4a7671199c695892c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 45819 } } peers { permanent_uuid: "c531986793ea429bbd0c9382b0232dd1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 34593 } }
I20260811 15:08:40.616406 5940 raft_consensus.cc:374] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:40.616475 5940 raft_consensus.cc:729] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1b43bc7c26bf43d4a7671199c695892c, State: Initialized, Role: FOLLOWER
I20260811 15:08:40.617003 5940 consensus_queue.cc:260] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0dafc7f5dcf4896840c7194b6ca46d6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 34989 } } peers { permanent_uuid: "1b43bc7c26bf43d4a7671199c695892c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 45819 } } peers { permanent_uuid: "c531986793ea429bbd0c9382b0232dd1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 34593 } }
I20260811 15:08:40.617658 5689 heartbeater.cc:503] Master 127.30.8.62:44883 was elected leader, sending a full tablet report...
I20260811 15:08:40.617992 5940 ts_tablet_manager.cc:1428] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c: Time spent starting tablet: real 0.005s user 0.003s sys 0.003s
I20260811 15:08:40.635403 5950 raft_consensus.cc:482] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:08:40.635650 5950 raft_consensus.cc:504] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0dafc7f5dcf4896840c7194b6ca46d6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 34989 } } peers { permanent_uuid: "1b43bc7c26bf43d4a7671199c695892c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 45819 } } peers { permanent_uuid: "c531986793ea429bbd0c9382b0232dd1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 34593 } }
I20260811 15:08:40.636868 5950 leader_election.cc:288] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e0dafc7f5dcf4896840c7194b6ca46d6 (127.30.8.1:34989), 1b43bc7c26bf43d4a7671199c695892c (127.30.8.2:45819)
I20260811 15:08:40.640455 5928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:40.637229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:54315 (local address 127.30.8.1:34989)
0811 15:08:40.637353 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:40.637357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:40.637371 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:08:40.637508 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:40.637511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:40.637554 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:40.637616 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:40.637875 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.638384 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.639518 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.639935 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.640030 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:40.640097 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:40.640262 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 15:08:40.640329 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 15:08:40.640455 5912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:40.637096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:34989 (local address 127.30.8.3:54315)
0811 15:08:40.637289 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:40.637305 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:40.637402 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:40.637636 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:40.637638 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:40.637655 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:40.637778 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:40.637783 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.638484 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.638488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:40.639402 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:40.639413 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.640044 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.640048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:40.640151 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:40.640170 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 15:08:40.640237 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 15:08:40.640314 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":33}
I20260811 15:08:40.640606 5930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:40.637274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:34817 (local address 127.30.8.2:45819)
0811 15:08:40.637398 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:40.637402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:40.637618 (+ 216us) server_negotiation.cc:407] Connection header received
0811 15:08:40.637730 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:40.637733 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:40.637778 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:40.637840 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:40.638087 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.638565 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.639662 (+ 1097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.640083 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.640173 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:40.640236 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:40.640417 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 15:08:40.640486 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 15:08:40.640627 5953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:40.637126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:45819 (local address 127.30.8.3:34817)
0811 15:08:40.637524 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:40.637535 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:40.637634 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:40.637862 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:40.637864 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:40.637879 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:40.637987 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:40.637992 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.638670 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.638673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:40.639543 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:40.639553 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.640195 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.640206 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:40.640305 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:40.640323 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 15:08:40.640386 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 15:08:40.640454 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"mutex_wait_us":17,"thread_start_us":139,"threads_started":1}
I20260811 15:08:40.641186 5399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc2378a174cb49c1bbbe62aee7d84f04" candidate_uuid: "c531986793ea429bbd0c9382b0232dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0dafc7f5dcf4896840c7194b6ca46d6" is_pre_election: true
I20260811 15:08:40.641228 5623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc2378a174cb49c1bbbe62aee7d84f04" candidate_uuid: "c531986793ea429bbd0c9382b0232dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b43bc7c26bf43d4a7671199c695892c" is_pre_election: true
I20260811 15:08:40.641505 5623 raft_consensus.cc:2454] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c531986793ea429bbd0c9382b0232dd1 in term 0.
I20260811 15:08:40.641505 5399 raft_consensus.cc:2454] T dc2378a174cb49c1bbbe62aee7d84f04 P e0dafc7f5dcf4896840c7194b6ca46d6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c531986793ea429bbd0c9382b0232dd1 in term 0.
I20260811 15:08:40.641987 5728 leader_election.cc:302] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c531986793ea429bbd0c9382b0232dd1, e0dafc7f5dcf4896840c7194b6ca46d6; no voters:
I20260811 15:08:40.642270 5950 raft_consensus.cc:2781] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:08:40.642351 5950 raft_consensus.cc:482] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:08:40.642385 5950 raft_consensus.cc:3037] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:40.643518 5950 raft_consensus.cc:504] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0dafc7f5dcf4896840c7194b6ca46d6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 34989 } } peers { permanent_uuid: "1b43bc7c26bf43d4a7671199c695892c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 45819 } } peers { permanent_uuid: "c531986793ea429bbd0c9382b0232dd1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 34593 } }
I20260811 15:08:40.643869 5950 leader_election.cc:288] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [CANDIDATE]: Term 1 election: Requested vote from peers e0dafc7f5dcf4896840c7194b6ca46d6 (127.30.8.1:34989), 1b43bc7c26bf43d4a7671199c695892c (127.30.8.2:45819)
I20260811 15:08:40.644251 5399 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc2378a174cb49c1bbbe62aee7d84f04" candidate_uuid: "c531986793ea429bbd0c9382b0232dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0dafc7f5dcf4896840c7194b6ca46d6"
I20260811 15:08:40.644253 5623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc2378a174cb49c1bbbe62aee7d84f04" candidate_uuid: "c531986793ea429bbd0c9382b0232dd1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1b43bc7c26bf43d4a7671199c695892c"
I20260811 15:08:40.644366 5623 raft_consensus.cc:3037] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:40.644374 5399 raft_consensus.cc:3037] T dc2378a174cb49c1bbbe62aee7d84f04 P e0dafc7f5dcf4896840c7194b6ca46d6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:40.645601 5399 raft_consensus.cc:2454] T dc2378a174cb49c1bbbe62aee7d84f04 P e0dafc7f5dcf4896840c7194b6ca46d6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c531986793ea429bbd0c9382b0232dd1 in term 1.
I20260811 15:08:40.645726 5623 raft_consensus.cc:2454] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c531986793ea429bbd0c9382b0232dd1 in term 1.
I20260811 15:08:40.645925 5728 leader_election.cc:302] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c531986793ea429bbd0c9382b0232dd1, e0dafc7f5dcf4896840c7194b6ca46d6; no voters:
I20260811 15:08:40.646122 5950 raft_consensus.cc:2781] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:40.646344 5950 raft_consensus.cc:686] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [term 1 LEADER]: Becoming Leader. State: Replica: c531986793ea429bbd0c9382b0232dd1, State: Running, Role: LEADER
I20260811 15:08:40.646677 5950 consensus_queue.cc:237] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0dafc7f5dcf4896840c7194b6ca46d6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 34989 } } peers { permanent_uuid: "1b43bc7c26bf43d4a7671199c695892c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 45819 } } peers { permanent_uuid: "c531986793ea429bbd0c9382b0232dd1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 34593 } }
I20260811 15:08:40.650056 5155 catalog_manager.cc:5495] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 reported cstate change: term changed from 0 to 1, leader changed from <none> to c531986793ea429bbd0c9382b0232dd1 (127.30.8.3). New cstate: current_term: 1 leader_uuid: "c531986793ea429bbd0c9382b0232dd1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e0dafc7f5dcf4896840c7194b6ca46d6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 34989 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1b43bc7c26bf43d4a7671199c695892c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 45819 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c531986793ea429bbd0c9382b0232dd1" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 34593 } health_report { overall_health: HEALTHY } } }
I20260811 15:08:40.668923 5936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:40.665775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52630 (local address 127.30.8.3:34593)
0811 15:08:40.665905 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:40.665908 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:40.665922 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:08:40.665999 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:40.666001 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:40.666059 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:40.666149 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:40.666338 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.666830 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.667616 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:40.667819 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:40.667891 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:40.668115 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:40.668578 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:40.668669 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:40.668697 (+ 28us) server_negotiation.cc:299] Negotiation successful
0811 15:08:40.668748 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 15:08:40.675324 5399 raft_consensus.cc:1261] T dc2378a174cb49c1bbbe62aee7d84f04 P e0dafc7f5dcf4896840c7194b6ca46d6 [term 1 FOLLOWER]: Refusing update from remote peer c531986793ea429bbd0c9382b0232dd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:40.675324 5623 raft_consensus.cc:1261] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c [term 1 FOLLOWER]: Refusing update from remote peer c531986793ea429bbd0c9382b0232dd1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:40.676069 5954 consensus_queue.cc:1035] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0dafc7f5dcf4896840c7194b6ca46d6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 34989 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:40.676260 5950 consensus_queue.cc:1035] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1b43bc7c26bf43d4a7671199c695892c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 45819 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:40.684212 5958 mvcc.cc:205] Tried to move back new op lower bound from 7317343931078610944 to 7317343930971906048. Current Snapshot: MvccSnapshot[applied={T|T < 7317343931078610944}]
I20260811 15:08:40.684816 5962 mvcc.cc:205] Tried to move back new op lower bound from 7317343931078610944 to 7317343930971906048. Current Snapshot: MvccSnapshot[applied={T|T < 7317343931078610944}]
I20260811 15:08:40.687994 5963 mvcc.cc:205] Tried to move back new op lower bound from 7317343931078610944 to 7317343930971906048. Current Snapshot: MvccSnapshot[applied={T|T < 7317343931078610944}]
I20260811 15:08:40.694218 5155 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:40.694406 5155 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:40.696828 5155 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1309710c884244a7adbf88df9ced4bfd: Sending DeleteTablet for 3 replicas of tablet dc2378a174cb49c1bbbe62aee7d84f04
I20260811 15:08:40.697712 5590 tablet_service.cc:1514] Processing DeleteTablet for tablet dc2378a174cb49c1bbbe62aee7d84f04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:40 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:42202
I20260811 15:08:40.697703 5801 tablet_service.cc:1514] Processing DeleteTablet for tablet dc2378a174cb49c1bbbe62aee7d84f04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:40 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:52614
I20260811 15:08:40.698457 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 5256
I20260811 15:08:40.698477 5978 tablet_replica.cc:330] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c: stopping tablet replica
I20260811 15:08:40.698490 5363 tablet_service.cc:1514] Processing DeleteTablet for tablet dc2378a174cb49c1bbbe62aee7d84f04 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:40 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:32844
I20260811 15:08:40.698836 5978 raft_consensus.cc:2229] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:40.699265 5978 raft_consensus.cc:2258] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:40.703317 5980 tablet_replica.cc:330] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1: stopping tablet replica
I20260811 15:08:40.703795 5980 raft_consensus.cc:2229] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:08:40.704296 5980 raft_consensus.cc:2258] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:40.711416 5980 ts_tablet_manager.cc:1905] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:40.713061 5978 ts_tablet_manager.cc:1905] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:40.714419 5980 ts_tablet_manager.cc:1918] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:08:40.714536 5980 log.cc:1196] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/dc2378a174cb49c1bbbe62aee7d84f04
I20260811 15:08:40.714875 5980 ts_tablet_manager.cc:1939] T dc2378a174cb49c1bbbe62aee7d84f04 P c531986793ea429bbd0c9382b0232dd1: Deleting consensus metadata
I20260811 15:08:40.715664 5978 ts_tablet_manager.cc:1918] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:08:40.715758 5978 log.cc:1196] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/dc2378a174cb49c1bbbe62aee7d84f04
I20260811 15:08:40.716053 5978 ts_tablet_manager.cc:1939] T dc2378a174cb49c1bbbe62aee7d84f04 P 1b43bc7c26bf43d4a7671199c695892c: Deleting consensus metadata
I20260811 15:08:40.716403 5135 catalog_manager.cc:4841] TS c531986793ea429bbd0c9382b0232dd1 (127.30.8.3:34593): tablet dc2378a174cb49c1bbbe62aee7d84f04 (table test-table [id=3a8a2add9c0f42ea9e8ed6620aa8d38b]) successfully deleted
I20260811 15:08:40.717176 5132 catalog_manager.cc:4841] TS 1b43bc7c26bf43d4a7671199c695892c (127.30.8.2:45819): tablet dc2378a174cb49c1bbbe62aee7d84f04 (table test-table [id=3a8a2add9c0f42ea9e8ed6620aa8d38b]) successfully deleted
W20260811 15:08:40.721951 5134 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.8.1:34989 (error 108)
I20260811 15:08:40.722936 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 5481
I20260811 15:08:40.723097 5929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:40.722761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:34989 (local address 127.0.0.1:32856)
0811 15:08:40.722896 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:40.723003 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.1:34989: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 15:08:40.760211 5134 catalog_manager.cc:4584] TS e0dafc7f5dcf4896840c7194b6ca46d6 (127.30.8.1:34989): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dc2378a174cb49c1bbbe62aee7d84f04: Network error: Client connection negotiation failed: client connection to 127.30.8.1:34989: connect: Connection refused (error 111)
I20260811 15:08:40.765822 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 5697
I20260811 15:08:40.781538 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 5109
2026-08-11T15:08:40Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2479 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T15:08:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:08:40Z Disabled control of system clock
I20260811 15:08:40.819597 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:33053
--webserver_interface=127.30.8.62
--webserver_port=0
--builtin_ntp_servers=127.30.8.20:41095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:33053
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 15:08:40.935738 5986 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:40.936040 5986 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:40.936084 5986 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:40.940217 5986 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:08:40.940300 5986 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:40.940315 5986 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:40.940335 5986 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:08:40.940364 5986 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:08:40.945387 5986 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:41095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:33053
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:33053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5986
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:40.946601 5986 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:40.948074 5986 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:40.956661 5997 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:40.956673 5992 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:40.956655 5994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:40.957057 5986 server_base.cc:1034] running on GCE node
I20260811 15:08:40.957610 5986 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:40.958788 5986 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:40.959940 5986 hybrid_clock.cc:648] HybridClock initialized: now 1786460920959917 us; error 45 us; skew 500 ppm
I20260811 15:08:40.961921 5986 webserver.cc:460] Webserver started at http://127.30.8.62:43231/ using document root <none> and password file <none>
I20260811 15:08:40.962517 5986 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:40.962585 5986 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:40.962817 5986 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:40.965034 5986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "5b748915ce724a1ead664a2fa017dfd1"
format_stamp: "Formatted at 2026-08-11 15:08:40 on dist-test-slave-1vmg"
server_key: "e7e4ede131a19f51ea264419833ec882"
server_key_iv: "95a81d768a83a2c41b0ca1884ac3e246"
server_key_version: "encryptionkey@0"
I20260811 15:08:40.965593 5986 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "5b748915ce724a1ead664a2fa017dfd1"
format_stamp: "Formatted at 2026-08-11 15:08:40 on dist-test-slave-1vmg"
server_key: "e7e4ede131a19f51ea264419833ec882"
server_key_iv: "95a81d768a83a2c41b0ca1884ac3e246"
server_key_version: "encryptionkey@0"
I20260811 15:08:40.969430 5986 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 15:08:40.971994 6008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:40.973254 5986 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:08:40.973397 5986 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "5b748915ce724a1ead664a2fa017dfd1"
format_stamp: "Formatted at 2026-08-11 15:08:40 on dist-test-slave-1vmg"
server_key: "e7e4ede131a19f51ea264419833ec882"
server_key_iv: "95a81d768a83a2c41b0ca1884ac3e246"
server_key_version: "encryptionkey@0"
I20260811 15:08:40.973510 5986 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:40.978410 5986 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:40.979092 5986 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:40.979264 5986 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:40.986119 5986 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:33053
I20260811 15:08:40.986161 6094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:33053 every 8 connection(s)
I20260811 15:08:40.987171 5986 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:08:40.990566 6096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:40.995232 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 5986
I20260811 15:08:40.995340 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:08:40.995597 30752 external_mini_cluster.cc:1428] Setting key cdcec7cb1b8bb57bc00c6e33a914e2a8
I20260811 15:08:40.997509 6096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1: Bootstrap starting.
I20260811 15:08:40.999943 6096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:41.000758 6096 log.cc:824] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:41.001382 6101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:40.997030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37880 (local address 127.30.8.62:33053)
0811 15:08:40.997722 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:40.997740 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:40.997778 (+ 38us) server_negotiation.cc:407] Connection header received
0811 15:08:40.998453 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:40.998472 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:40.998844 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:40.999210 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:41.000011 (+ 801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:41.000016 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:41.000028 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:08:41.000041 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:41.000115 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:08:41.000138 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:41.000245 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:41.000373 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 15:08:41.000631 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":137,"threads_started":1}
I20260811 15:08:41.002899 6096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1: No bootstrap required, opened a new log
I20260811 15:08:41.006685 6096 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b748915ce724a1ead664a2fa017dfd1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 33053 } }
I20260811 15:08:41.006978 6096 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:41.007050 6096 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5b748915ce724a1ead664a2fa017dfd1, State: Initialized, Role: FOLLOWER
I20260811 15:08:41.007504 6096 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b748915ce724a1ead664a2fa017dfd1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 33053 } }
I20260811 15:08:41.007639 6096 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:08:41.007709 6096 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:08:41.007808 6096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:41.008934 6096 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b748915ce724a1ead664a2fa017dfd1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 33053 } }
I20260811 15:08:41.009297 6096 leader_election.cc:302] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5b748915ce724a1ead664a2fa017dfd1; no voters:
I20260811 15:08:41.009629 6096 leader_election.cc:288] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:08:41.009770 6105 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:41.010017 6105 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1 [term 1 LEADER]: Becoming Leader. State: Replica: 5b748915ce724a1ead664a2fa017dfd1, State: Running, Role: LEADER
I20260811 15:08:41.010339 6105 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b748915ce724a1ead664a2fa017dfd1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 33053 } }
I20260811 15:08:41.011067 6096 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:08:41.012022 6108 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5b748915ce724a1ead664a2fa017dfd1. Latest consensus state: current_term: 1 leader_uuid: "5b748915ce724a1ead664a2fa017dfd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b748915ce724a1ead664a2fa017dfd1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 33053 } } }
I20260811 15:08:41.012171 6108 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:41.012421 6107 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5b748915ce724a1ead664a2fa017dfd1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5b748915ce724a1ead664a2fa017dfd1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 33053 } } }
I20260811 15:08:41.012501 6107 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:41.012509 6117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:08:41.015507 6117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:08:41.020980 6117 catalog_manager.cc:1347] Generated new cluster ID: 3d5cc81b34924c2980501ede1ce640e5
I20260811 15:08:41.021083 6117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:08:41.049221 6117 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:08:41.049572 6117 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:08:41.064200 6117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1: Generated new TSK 0
I20260811 15:08:41.065004 6117 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:08:41.069190 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:33053
--builtin_ntp_servers=127.30.8.20:41095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 15:08:41.189833 6140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:41.190136 6140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:41.190187 6140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:41.194288 6140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:41.194370 6140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:41.194440 6140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:08:41.199436 6140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:41095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:33053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:41.200702 6140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:41.202162 6140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:41.211995 6148 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:41.212239 6140 server_base.cc:1034] running on GCE node
W20260811 15:08:41.212005 6151 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:41.212013 6149 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:41.212790 6140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:41.213481 6140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:41.214643 6140 hybrid_clock.cc:648] HybridClock initialized: now 1786460921214632 us; error 32 us; skew 500 ppm
I20260811 15:08:41.216631 6140 webserver.cc:460] Webserver started at http://127.30.8.1:42583/ using document root <none> and password file <none>
I20260811 15:08:41.217226 6140 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:41.217293 6140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:41.217507 6140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:41.219712 6140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "6b088082407e4768be722a5787dea2c2"
format_stamp: "Formatted at 2026-08-11 15:08:41 on dist-test-slave-1vmg"
server_key: "be100c30ea8bbf03a329d3cf994de6f9"
server_key_iv: "8b06c9b44756077394b8b4bbb08c0871"
server_key_version: "encryptionkey@0"
I20260811 15:08:41.220284 6140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "6b088082407e4768be722a5787dea2c2"
format_stamp: "Formatted at 2026-08-11 15:08:41 on dist-test-slave-1vmg"
server_key: "be100c30ea8bbf03a329d3cf994de6f9"
server_key_iv: "8b06c9b44756077394b8b4bbb08c0871"
server_key_version: "encryptionkey@0"
I20260811 15:08:41.224084 6140 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 15:08:41.226670 6164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:41.227989 6140 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 15:08:41.228109 6140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "6b088082407e4768be722a5787dea2c2"
format_stamp: "Formatted at 2026-08-11 15:08:41 on dist-test-slave-1vmg"
server_key: "be100c30ea8bbf03a329d3cf994de6f9"
server_key_iv: "8b06c9b44756077394b8b4bbb08c0871"
server_key_version: "encryptionkey@0"
I20260811 15:08:41.228224 6140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:41.233018 6140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:41.233651 6140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:41.233824 6140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:41.234368 6140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:41.235384 6140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:41.235433 6140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:41.235486 6140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:41.235515 6140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:41.244236 6140 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:42547
I20260811 15:08:41.244264 6345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:42547 every 8 connection(s)
I20260811 15:08:41.245195 6140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:08:41.246315 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 6140
I20260811 15:08:41.246426 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:08:41.246744 30752 external_mini_cluster.cc:1428] Setting key 943a261ac0a195298903f9e5b367ccd3
I20260811 15:08:41.249202 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:33053
--builtin_ntp_servers=127.30.8.20:41095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 15:08:41.252336 6101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:41.247247 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:58517 (local address 127.30.8.62:33053)
0811 15:08:41.247386 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:41.247391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:41.248632 (+ 1241us) server_negotiation.cc:407] Connection header received
0811 15:08:41.250452 (+ 1820us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:41.250455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:41.250495 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:41.250563 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:41.251876 (+ 1313us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:41.251881 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:41.251884 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:08:41.251899 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:41.251925 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:08:41.251936 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:41.252040 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:41.252147 (+ 107us) server_negotiation.cc:299] Negotiation successful
0811 15:08:41.252195 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 15:08:41.253234 6351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:41.247656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:33053 (local address 127.30.8.1:58517)
0811 15:08:41.248466 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:41.248524 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:41.250200 (+ 1676us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:41.250785 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:41.250794 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:41.250945 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:41.251582 (+ 637us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:08:41.251584 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:41.251627 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:08:41.251712 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:41.252022 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:41.252027 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:41.252031 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 15:08:41.252165 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 15:08:41.252381 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":154,"threads_started":1}
I20260811 15:08:41.254542 6346 heartbeater.cc:347] Connected to a master server at 127.30.8.62:33053
I20260811 15:08:41.254796 6346 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:41.255280 6346 heartbeater.cc:511] Master 127.30.8.62:33053 requested a full tablet report, sending...
I20260811 15:08:41.256894 6038 ts_manager.cc:194] Registered new tserver with Master: 6b088082407e4768be722a5787dea2c2 (127.30.8.1:42547)
W20260811 15:08:41.369761 6352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:41.370059 6352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:41.370110 6352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:41.374239 6352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:41.374325 6352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:41.374390 6352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:08:41.379457 6352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:41095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:33053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6352
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:41.380710 6352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:41.382159 6352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:41.392019 6362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:41.392198 6352 server_base.cc:1034] running on GCE node
W20260811 15:08:41.392071 6357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:41.392030 6360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:41.392751 6352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:41.393436 6352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:41.394603 6352 hybrid_clock.cc:648] HybridClock initialized: now 1786460921394582 us; error 36 us; skew 500 ppm
I20260811 15:08:41.396544 6352 webserver.cc:460] Webserver started at http://127.30.8.2:40459/ using document root <none> and password file <none>
I20260811 15:08:41.397149 6352 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:41.397215 6352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:41.397436 6352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:41.399643 6352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "341cf2df84864b22b9c64432faaa023e"
format_stamp: "Formatted at 2026-08-11 15:08:41 on dist-test-slave-1vmg"
server_key: "2c0d498360501e6e47234cb178cb2940"
server_key_iv: "44b08006a98a0f9a78d97e7fa38f0be4"
server_key_version: "encryptionkey@0"
I20260811 15:08:41.400233 6352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "341cf2df84864b22b9c64432faaa023e"
format_stamp: "Formatted at 2026-08-11 15:08:41 on dist-test-slave-1vmg"
server_key: "2c0d498360501e6e47234cb178cb2940"
server_key_iv: "44b08006a98a0f9a78d97e7fa38f0be4"
server_key_version: "encryptionkey@0"
I20260811 15:08:41.404084 6352 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:08:41.406703 6374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:41.408099 6352 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:08:41.408229 6352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "341cf2df84864b22b9c64432faaa023e"
format_stamp: "Formatted at 2026-08-11 15:08:41 on dist-test-slave-1vmg"
server_key: "2c0d498360501e6e47234cb178cb2940"
server_key_iv: "44b08006a98a0f9a78d97e7fa38f0be4"
server_key_version: "encryptionkey@0"
I20260811 15:08:41.408357 6352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:41.413483 6352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:41.414122 6352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:41.414268 6352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:41.414795 6352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:41.415850 6352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:41.415911 6352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:41.415959 6352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:41.415980 6352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:41.424619 6352 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:34607
I20260811 15:08:41.424639 6545 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:34607 every 8 connection(s)
I20260811 15:08:41.425602 6352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:08:41.427006 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 6352
I20260811 15:08:41.427124 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:08:41.427397 30752 external_mini_cluster.cc:1428] Setting key 062763a94a7a34446d09669b52e1036a
I20260811 15:08:41.429435 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:33053
--builtin_ntp_servers=127.30.8.20:41095
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 15:08:41.432205 6101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:41.427556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:53901 (local address 127.30.8.62:33053)
0811 15:08:41.427728 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:41.427733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:41.429042 (+ 1309us) server_negotiation.cc:407] Connection header received
0811 15:08:41.430391 (+ 1349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:41.430395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:41.430441 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:41.430524 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:41.431769 (+ 1245us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:41.431773 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:41.431775 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:08:41.431787 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:41.431808 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:08:41.431816 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:41.431894 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:41.432037 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 15:08:41.432076 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 15:08:41.433295 6552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:41.428075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:33053 (local address 127.30.8.2:53901)
0811 15:08:41.428889 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:41.428937 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:41.430158 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:41.430680 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:41.430687 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:41.430777 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:41.431441 (+ 664us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:08:41.431443 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:41.431503 (+ 60us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:08:41.431606 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:41.431898 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:41.431906 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:41.431912 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 15:08:41.432086 (+ 174us) client_negotiation.cc:240] Negotiation successful
0811 15:08:41.432488 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":108,"threads_started":1}
I20260811 15:08:41.434535 6547 heartbeater.cc:347] Connected to a master server at 127.30.8.62:33053
I20260811 15:08:41.434832 6547 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:41.435410 6547 heartbeater.cc:511] Master 127.30.8.62:33053 requested a full tablet report, sending...
I20260811 15:08:41.436653 6038 ts_manager.cc:194] Registered new tserver with Master: 341cf2df84864b22b9c64432faaa023e (127.30.8.2:34607)
W20260811 15:08:41.551038 6553 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:41.551331 6553 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:41.551373 6553 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:41.555639 6553 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:41.555714 6553 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:41.555788 6553 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:08:41.560854 6553 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:41095
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:33053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6553
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:41.562060 6553 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:41.563594 6553 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:41.573588 6566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:41.573787 6553 server_base.cc:1034] running on GCE node
W20260811 15:08:41.573587 6561 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:41.573997 6562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:41.574401 6553 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:41.575162 6553 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:41.576333 6553 hybrid_clock.cc:648] HybridClock initialized: now 1786460921576317 us; error 31 us; skew 500 ppm
I20260811 15:08:41.578299 6553 webserver.cc:460] Webserver started at http://127.30.8.3:35377/ using document root <none> and password file <none>
I20260811 15:08:41.578902 6553 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:41.579036 6553 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:41.579273 6553 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:41.581413 6553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "5724aa62f94c4ed2a5af861ed2e31f92"
format_stamp: "Formatted at 2026-08-11 15:08:41 on dist-test-slave-1vmg"
server_key: "68cb3e4df34dc8604afac3bbf776596f"
server_key_iv: "7bb6622727c06de84e4ce0c976bfc2fc"
server_key_version: "encryptionkey@0"
I20260811 15:08:41.581954 6553 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "5724aa62f94c4ed2a5af861ed2e31f92"
format_stamp: "Formatted at 2026-08-11 15:08:41 on dist-test-slave-1vmg"
server_key: "68cb3e4df34dc8604afac3bbf776596f"
server_key_iv: "7bb6622727c06de84e4ce0c976bfc2fc"
server_key_version: "encryptionkey@0"
I20260811 15:08:41.585860 6553 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 15:08:41.588815 6575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:41.590152 6553 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 15:08:41.590277 6553 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "5724aa62f94c4ed2a5af861ed2e31f92"
format_stamp: "Formatted at 2026-08-11 15:08:41 on dist-test-slave-1vmg"
server_key: "68cb3e4df34dc8604afac3bbf776596f"
server_key_iv: "7bb6622727c06de84e4ce0c976bfc2fc"
server_key_version: "encryptionkey@0"
I20260811 15:08:41.590399 6553 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:41.595242 6553 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:41.595901 6553 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:41.596074 6553 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:41.596642 6553 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:41.597610 6553 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:41.597661 6553 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:41.597715 6553 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:41.597745 6553 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:41.606511 6761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:44807 every 8 connection(s)
I20260811 15:08:41.606490 6553 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:44807
I20260811 15:08:41.607614 6553 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:08:41.613883 6101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:41.609562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:41603 (local address 127.30.8.62:33053)
0811 15:08:41.609740 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:41.609745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:41.610802 (+ 1057us) server_negotiation.cc:407] Connection header received
0811 15:08:41.612193 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:41.612197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:41.612245 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:41.612326 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:41.613498 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:41.613501 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:41.613503 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:08:41.613515 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:41.613537 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:08:41.613545 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:41.613626 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:41.613726 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 15:08:41.613766 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 15:08:41.614791 6764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:41.609984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:33053 (local address 127.30.8.3:41603)
0811 15:08:41.610651 (+ 667us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:41.610691 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:41.611950 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:41.612515 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:41.612523 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:41.612615 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:41.613242 (+ 627us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:08:41.613243 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:41.613286 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:08:41.613358 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:41.613617 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:41.613621 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:41.613625 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 15:08:41.613754 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 15:08:41.613969 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":108,"threads_started":1}
I20260811 15:08:41.615959 6762 heartbeater.cc:347] Connected to a master server at 127.30.8.62:33053
I20260811 15:08:41.616187 6762 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:41.616596 6762 heartbeater.cc:511] Master 127.30.8.62:33053 requested a full tablet report, sending...
I20260811 15:08:41.617523 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 6553
I20260811 15:08:41.617615 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:08:41.617658 6038 ts_manager.cc:194] Registered new tserver with Master: 5724aa62f94c4ed2a5af861ed2e31f92 (127.30.8.3:44807)
I20260811 15:08:41.617921 30752 external_mini_cluster.cc:1428] Setting key 42e11467d967e24a60d0e991dd5c7345
I20260811 15:08:41.619936 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 15:08:41.623328 6101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:41.621767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37886 (local address 127.30.8.62:33053)
0811 15:08:41.621880 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:41.621885 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:41.622064 (+ 179us) server_negotiation.cc:407] Connection header received
0811 15:08:41.622177 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:41.622180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:41.622217 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:41.622282 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:41.622953 (+ 671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:41.622957 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:41.622960 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:08:41.622972 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:41.622994 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:08:41.623003 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:41.623075 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:41.623155 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 15:08:41.623198 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 15:08:41.626005 6038 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37886:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:08:41.629113 6038 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:08:41.641301 6789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:41.638484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:42547 (local address 127.0.0.1:54554)
0811 15:08:41.639262 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:41.639272 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:41.639353 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:41.639757 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:41.639764 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:41.639843 (+ 79us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:41.640440 (+ 597us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:08:41.640442 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:41.640465 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:08:41.640536 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:41.640940 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:41.640950 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:41.640956 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 15:08:41.641071 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 15:08:41.641141 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"thread_start_us":95,"threads_started":1}
I20260811 15:08:41.641382 6791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:41.638653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54554 (local address 127.30.8.1:42547)
0811 15:08:41.639122 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:41.639133 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:41.639366 (+ 233us) server_negotiation.cc:407] Connection header received
0811 15:08:41.639429 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:41.639435 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:41.639592 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:41.639729 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:41.640681 (+ 952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:41.640688 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:41.640703 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:08:41.640715 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:41.640762 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:08:41.640799 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:41.640899 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:41.641126 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 15:08:41.641220 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":161,"threads_started":1}
I20260811 15:08:41.641558 6781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:41.637856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:44807 (local address 127.0.0.1:44356)
0811 15:08:41.638582 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:41.638595 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:41.638671 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:41.640554 (+ 1883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:41.640557 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:41.640570 (+ 13us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:41.640939 (+ 369us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:08:41.640941 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:41.640951 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:08:41.640970 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:41.641346 (+ 376us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:41.641350 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:41.641354 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 15:08:41.641412 (+ 58us) client_negotiation.cc:240] Negotiation successful
0811 15:08:41.641445 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":120,"threads_started":1}
I20260811 15:08:41.641949 6780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:41.637742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:34607 (local address 127.0.0.1:56534)
0811 15:08:41.638116 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:41.638165 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:41.638269 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:41.640304 (+ 2035us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:41.640307 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:41.640322 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:41.640788 (+ 466us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:08:41.640790 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:41.640800 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:08:41.640850 (+ 50us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:41.641343 (+ 493us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:41.641346 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:41.641349 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 15:08:41.641808 (+ 459us) client_negotiation.cc:240] Negotiation successful
0811 15:08:41.641843 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":86,"threads_started":1}
I20260811 15:08:41.642007 6782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:41.638156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44356 (local address 127.30.8.3:44807)
0811 15:08:41.640151 (+ 1995us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:41.640163 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:41.640185 (+ 22us) server_negotiation.cc:407] Connection header received
0811 15:08:41.640235 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:41.640240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:41.640402 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:41.640576 (+ 174us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:41.641145 (+ 569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:41.641150 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:41.641161 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:08:41.641172 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:41.641211 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:08:41.641236 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:41.641751 (+ 515us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:41.641797 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 15:08:41.641867 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1889,"thread_start_us":121,"threads_started":1}
I20260811 15:08:41.642153 6790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:41.638439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56534 (local address 127.30.8.2:34607)
0811 15:08:41.638952 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:41.638964 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:41.638992 (+ 28us) server_negotiation.cc:407] Connection header received
0811 15:08:41.639048 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:41.639054 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:41.639240 (+ 186us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:41.639387 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:41.641105 (+ 1718us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:41.641111 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:41.641122 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:08:41.641133 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:41.641172 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:08:41.641206 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:41.641300 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:41.641845 (+ 545us) server_negotiation.cc:299] Negotiation successful
0811 15:08:41.641902 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":128,"threads_started":1}
I20260811 15:08:41.644876 6239 tablet_service.cc:1467] Processing CreateTablet for tablet 49f7b41be9cf45a0baeebcc337ec4cb8 (DEFAULT_TABLE table=test-table [id=716a5983a37946e293ed65f214992c8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:41.645444 6442 tablet_service.cc:1467] Processing CreateTablet for tablet 49f7b41be9cf45a0baeebcc337ec4cb8 (DEFAULT_TABLE table=test-table [id=716a5983a37946e293ed65f214992c8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:41.645588 6655 tablet_service.cc:1467] Processing CreateTablet for tablet 49f7b41be9cf45a0baeebcc337ec4cb8 (DEFAULT_TABLE table=test-table [id=716a5983a37946e293ed65f214992c8b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:41.646144 6239 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49f7b41be9cf45a0baeebcc337ec4cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:41.646409 6442 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49f7b41be9cf45a0baeebcc337ec4cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:41.646535 6655 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 49f7b41be9cf45a0baeebcc337ec4cb8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:41.652841 6795 tablet_bootstrap.cc:492] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92: Bootstrap starting.
I20260811 15:08:41.654042 6797 tablet_bootstrap.cc:492] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e: Bootstrap starting.
I20260811 15:08:41.655287 6796 tablet_bootstrap.cc:492] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2: Bootstrap starting.
I20260811 15:08:41.655619 6795 tablet_bootstrap.cc:654] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:41.655903 6797 tablet_bootstrap.cc:654] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:41.656497 6795 log.cc:824] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:41.656709 6797 log.cc:824] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:41.657287 6796 tablet_bootstrap.cc:654] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:41.658103 6796 log.cc:824] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:41.658726 6795 tablet_bootstrap.cc:492] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92: No bootstrap required, opened a new log
I20260811 15:08:41.658753 6797 tablet_bootstrap.cc:492] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e: No bootstrap required, opened a new log
I20260811 15:08:41.658951 6795 ts_tablet_manager.cc:1397] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.002s
I20260811 15:08:41.658974 6797 ts_tablet_manager.cc:1397] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e: Time spent bootstrapping tablet: real 0.005s user 0.005s sys 0.000s
I20260811 15:08:41.660573 6796 tablet_bootstrap.cc:492] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2: No bootstrap required, opened a new log
I20260811 15:08:41.660779 6796 ts_tablet_manager.cc:1397] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260811 15:08:41.662822 6797 raft_consensus.cc:348] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "341cf2df84864b22b9c64432faaa023e" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 34607 } } peers { permanent_uuid: "5724aa62f94c4ed2a5af861ed2e31f92" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44807 } } peers { permanent_uuid: "6b088082407e4768be722a5787dea2c2" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 42547 } }
I20260811 15:08:41.663108 6797 raft_consensus.cc:374] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:41.663166 6797 raft_consensus.cc:729] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 341cf2df84864b22b9c64432faaa023e, State: Initialized, Role: FOLLOWER
I20260811 15:08:41.663321 6795 raft_consensus.cc:348] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "341cf2df84864b22b9c64432faaa023e" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 34607 } } peers { permanent_uuid: "5724aa62f94c4ed2a5af861ed2e31f92" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44807 } } peers { permanent_uuid: "6b088082407e4768be722a5787dea2c2" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 42547 } }
I20260811 15:08:41.663573 6795 raft_consensus.cc:374] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:41.663635 6795 raft_consensus.cc:729] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5724aa62f94c4ed2a5af861ed2e31f92, State: Initialized, Role: FOLLOWER
I20260811 15:08:41.663633 6797 consensus_queue.cc:260] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "341cf2df84864b22b9c64432faaa023e" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 34607 } } peers { permanent_uuid: "5724aa62f94c4ed2a5af861ed2e31f92" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44807 } } peers { permanent_uuid: "6b088082407e4768be722a5787dea2c2" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 42547 } }
I20260811 15:08:41.664115 6795 consensus_queue.cc:260] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "341cf2df84864b22b9c64432faaa023e" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 34607 } } peers { permanent_uuid: "5724aa62f94c4ed2a5af861ed2e31f92" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44807 } } peers { permanent_uuid: "6b088082407e4768be722a5787dea2c2" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 42547 } }
I20260811 15:08:41.664418 6547 heartbeater.cc:503] Master 127.30.8.62:33053 was elected leader, sending a full tablet report...
I20260811 15:08:41.664717 6797 ts_tablet_manager.cc:1428] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 15:08:41.665035 6795 ts_tablet_manager.cc:1428] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92: Time spent starting tablet: real 0.006s user 0.003s sys 0.002s
I20260811 15:08:41.665139 6796 raft_consensus.cc:348] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "341cf2df84864b22b9c64432faaa023e" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 34607 } } peers { permanent_uuid: "5724aa62f94c4ed2a5af861ed2e31f92" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44807 } } peers { permanent_uuid: "6b088082407e4768be722a5787dea2c2" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 42547 } }
I20260811 15:08:41.665385 6796 raft_consensus.cc:374] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:41.665395 6762 heartbeater.cc:503] Master 127.30.8.62:33053 was elected leader, sending a full tablet report...
I20260811 15:08:41.665443 6796 raft_consensus.cc:729] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b088082407e4768be722a5787dea2c2, State: Initialized, Role: FOLLOWER
I20260811 15:08:41.665877 6796 consensus_queue.cc:260] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "341cf2df84864b22b9c64432faaa023e" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 34607 } } peers { permanent_uuid: "5724aa62f94c4ed2a5af861ed2e31f92" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44807 } } peers { permanent_uuid: "6b088082407e4768be722a5787dea2c2" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 42547 } }
I20260811 15:08:41.666548 6346 heartbeater.cc:503] Master 127.30.8.62:33053 was elected leader, sending a full tablet report...
I20260811 15:08:41.666781 6796 ts_tablet_manager.cc:1428] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260811 15:08:41.676594 6550 tablet.cc:2369] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 15:08:41.746426 6349 tablet.cc:2369] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:08:41.785538 6803 raft_consensus.cc:482] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:08:41.785785 6803 raft_consensus.cc:504] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "341cf2df84864b22b9c64432faaa023e" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 34607 } } peers { permanent_uuid: "5724aa62f94c4ed2a5af861ed2e31f92" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44807 } } peers { permanent_uuid: "6b088082407e4768be722a5787dea2c2" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 42547 } }
I20260811 15:08:41.786979 6803 leader_election.cc:288] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 341cf2df84864b22b9c64432faaa023e (127.30.8.2:34607), 5724aa62f94c4ed2a5af861ed2e31f92 (127.30.8.3:44807)
I20260811 15:08:41.789146 6782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:41.787403 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:54935 (local address 127.30.8.3:44807)
0811 15:08:41.787687 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:41.787691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:41.787708 (+ 17us) server_negotiation.cc:407] Connection header received
0811 15:08:41.787828 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:41.787831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:41.787867 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:41.787935 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:41.788679 (+ 744us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:41.788683 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:41.788686 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:08:41.788698 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:41.788722 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:08:41.788734 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:41.788834 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:41.788977 (+ 143us) server_negotiation.cc:299] Negotiation successful
0811 15:08:41.789017 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 15:08:41.789341 6790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:41.787401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:44163 (local address 127.30.8.2:34607)
0811 15:08:41.787557 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:41.787562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:41.787963 (+ 401us) server_negotiation.cc:407] Connection header received
0811 15:08:41.788064 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:41.788067 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:41.788103 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:41.788171 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:41.788953 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:41.788956 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:41.788958 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:08:41.788972 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:41.788993 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:08:41.789003 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:41.789084 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:41.789190 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 15:08:41.789232 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 15:08:41.789160 6804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:41.787270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:44807 (local address 127.30.8.1:54935)
0811 15:08:41.787592 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:41.787609 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:41.787717 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:41.787982 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:41.787986 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:41.788002 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:41.788485 (+ 483us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:08:41.788488 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:41.788501 (+ 13us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:08:41.788533 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:41.788856 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:41.788860 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:41.788864 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 15:08:41.788964 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 15:08:41.789002 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":94,"threads_started":1}
I20260811 15:08:41.789613 6805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:41.787270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:34607 (local address 127.30.8.1:44163)
0811 15:08:41.787859 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:41.787870 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:41.787959 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:41.788204 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:41.788207 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:41.788221 (+ 14us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:41.788773 (+ 552us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:08:41.788775 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:41.788785 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:08:41.788812 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:41.789088 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:41.789091 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:41.789094 (+ 3us) client_negotiation.cc:769] Sending connection context
0811 15:08:41.789174 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 15:08:41.789210 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":529,"mutex_wait_us":308,"thread_start_us":61,"threads_started":1}
I20260811 15:08:41.790062 6681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f7b41be9cf45a0baeebcc337ec4cb8" candidate_uuid: "6b088082407e4768be722a5787dea2c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5724aa62f94c4ed2a5af861ed2e31f92" is_pre_election: true
I20260811 15:08:41.790272 6478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f7b41be9cf45a0baeebcc337ec4cb8" candidate_uuid: "6b088082407e4768be722a5787dea2c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "341cf2df84864b22b9c64432faaa023e" is_pre_election: true
I20260811 15:08:41.790407 6681 raft_consensus.cc:2454] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b088082407e4768be722a5787dea2c2 in term 0.
I20260811 15:08:41.790556 6478 raft_consensus.cc:2454] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6b088082407e4768be722a5787dea2c2 in term 0.
I20260811 15:08:41.790979 6168 leader_election.cc:302] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 341cf2df84864b22b9c64432faaa023e, 5724aa62f94c4ed2a5af861ed2e31f92, 6b088082407e4768be722a5787dea2c2; no voters:
I20260811 15:08:41.791290 6803 raft_consensus.cc:2781] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:08:41.791380 6803 raft_consensus.cc:482] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:08:41.791429 6803 raft_consensus.cc:3037] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:41.792466 6803 raft_consensus.cc:504] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "341cf2df84864b22b9c64432faaa023e" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 34607 } } peers { permanent_uuid: "5724aa62f94c4ed2a5af861ed2e31f92" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44807 } } peers { permanent_uuid: "6b088082407e4768be722a5787dea2c2" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 42547 } }
I20260811 15:08:41.792793 6803 leader_election.cc:288] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2 [CANDIDATE]: Term 1 election: Requested vote from peers 341cf2df84864b22b9c64432faaa023e (127.30.8.2:34607), 5724aa62f94c4ed2a5af861ed2e31f92 (127.30.8.3:44807)
I20260811 15:08:41.793131 6478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f7b41be9cf45a0baeebcc337ec4cb8" candidate_uuid: "6b088082407e4768be722a5787dea2c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "341cf2df84864b22b9c64432faaa023e"
I20260811 15:08:41.793159 6681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "49f7b41be9cf45a0baeebcc337ec4cb8" candidate_uuid: "6b088082407e4768be722a5787dea2c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5724aa62f94c4ed2a5af861ed2e31f92"
I20260811 15:08:41.793251 6478 raft_consensus.cc:3037] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:41.793287 6681 raft_consensus.cc:3037] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:41.794324 6681 raft_consensus.cc:2454] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b088082407e4768be722a5787dea2c2 in term 1.
I20260811 15:08:41.794346 6478 raft_consensus.cc:2454] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6b088082407e4768be722a5787dea2c2 in term 1.
I20260811 15:08:41.794636 6166 leader_election.cc:302] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 341cf2df84864b22b9c64432faaa023e, 6b088082407e4768be722a5787dea2c2; no voters:
I20260811 15:08:41.794817 6803 raft_consensus.cc:2781] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:41.795070 6803 raft_consensus.cc:686] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2 [term 1 LEADER]: Becoming Leader. State: Replica: 6b088082407e4768be722a5787dea2c2, State: Running, Role: LEADER
I20260811 15:08:41.795414 6803 consensus_queue.cc:237] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "341cf2df84864b22b9c64432faaa023e" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 34607 } } peers { permanent_uuid: "5724aa62f94c4ed2a5af861ed2e31f92" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44807 } } peers { permanent_uuid: "6b088082407e4768be722a5787dea2c2" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 42547 } }
I20260811 15:08:41.798861 6032 catalog_manager.cc:5495] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6b088082407e4768be722a5787dea2c2 (127.30.8.1). New cstate: current_term: 1 leader_uuid: "6b088082407e4768be722a5787dea2c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "341cf2df84864b22b9c64432faaa023e" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 34607 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5724aa62f94c4ed2a5af861ed2e31f92" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44807 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b088082407e4768be722a5787dea2c2" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 42547 } health_report { overall_health: HEALTHY } } }
I20260811 15:08:41.833547 6791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:41.832123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54564 (local address 127.30.8.1:42547)
0811 15:08:41.832271 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:41.832275 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:41.832288 (+ 13us) server_negotiation.cc:407] Connection header received
0811 15:08:41.832371 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:41.832373 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:41.832437 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:41.832505 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:41.833164 (+ 659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:41.833167 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:41.833169 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:08:41.833180 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:41.833200 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:08:41.833208 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:41.833281 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:41.833373 (+ 92us) server_negotiation.cc:299] Negotiation successful
0811 15:08:41.833408 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 15:08:41.839824 6681 raft_consensus.cc:1261] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92 [term 1 FOLLOWER]: Refusing update from remote peer 6b088082407e4768be722a5787dea2c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:41.839824 6478 raft_consensus.cc:1261] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e [term 1 FOLLOWER]: Refusing update from remote peer 6b088082407e4768be722a5787dea2c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:41.840603 6803 consensus_queue.cc:1035] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "341cf2df84864b22b9c64432faaa023e" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 34607 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:41.840821 6806 consensus_queue.cc:1035] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 6b088082407e4768be722a5787dea2c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5724aa62f94c4ed2a5af861ed2e31f92" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:41.848456 6810 mvcc.cc:205] Tried to move back new op lower bound from 7317343935847837696 to 7317343935676649472. Current Snapshot: MvccSnapshot[applied={T|T < 7317343935847837696}]
I20260811 15:08:41.858000 6032 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:41.858222 6032 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37886:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:41.860869 6032 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5b748915ce724a1ead664a2fa017dfd1: Sending DeleteTablet for 3 replicas of tablet 49f7b41be9cf45a0baeebcc337ec4cb8
I20260811 15:08:41.861821 6655 tablet_service.cc:1514] Processing DeleteTablet for tablet 49f7b41be9cf45a0baeebcc337ec4cb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:41 UTC) from {username='slave'} at 127.0.0.1:44356
I20260811 15:08:41.861838 6442 tablet_service.cc:1514] Processing DeleteTablet for tablet 49f7b41be9cf45a0baeebcc337ec4cb8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:41 UTC) from {username='slave'} at 127.0.0.1:56534
I20260811 15:08:41.862279 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 6140
I20260811 15:08:41.870973 6832 tablet_replica.cc:330] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e: stopping tablet replica
I20260811 15:08:41.870973 6831 tablet_replica.cc:330] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92: stopping tablet replica
I20260811 15:08:41.871403 6832 raft_consensus.cc:2229] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:41.871403 6831 raft_consensus.cc:2229] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:41.871817 6832 raft_consensus.cc:2258] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:41.871817 6831 raft_consensus.cc:2258] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:41.873368 6832 ts_tablet_manager.cc:1905] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:41.875958 6832 ts_tablet_manager.cc:1918] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:08:41.876076 6832 log.cc:1196] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/49f7b41be9cf45a0baeebcc337ec4cb8
I20260811 15:08:41.876359 6832 ts_tablet_manager.cc:1939] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 341cf2df84864b22b9c64432faaa023e: Deleting consensus metadata
I20260811 15:08:41.876927 6831 ts_tablet_manager.cc:1905] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:41.877766 6011 catalog_manager.cc:4841] TS 341cf2df84864b22b9c64432faaa023e (127.30.8.2:34607): tablet 49f7b41be9cf45a0baeebcc337ec4cb8 (table test-table [id=716a5983a37946e293ed65f214992c8b]) successfully deleted
W20260811 15:08:41.878191 6011 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.8.1:42547 (error 108)
I20260811 15:08:41.878464 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 6352
I20260811 15:08:41.882115 6831 ts_tablet_manager.cc:1918] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:08:41.882246 6831 log.cc:1196] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/49f7b41be9cf45a0baeebcc337ec4cb8
I20260811 15:08:41.882601 6831 ts_tablet_manager.cc:1939] T 49f7b41be9cf45a0baeebcc337ec4cb8 P 5724aa62f94c4ed2a5af861ed2e31f92: Deleting consensus metadata
I20260811 15:08:41.884402 6013 catalog_manager.cc:4841] TS 5724aa62f94c4ed2a5af861ed2e31f92 (127.30.8.3:44807): tablet 49f7b41be9cf45a0baeebcc337ec4cb8 (table test-table [id=716a5983a37946e293ed65f214992c8b]) successfully deleted
I20260811 15:08:41.887014 6789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:41.886596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:42547 (local address 127.0.0.1:54568)
0811 15:08:41.886783 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:41.886880 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.1:42547: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260811 15:08:41.887213 6011 catalog_manager.cc:4584] TS 6b088082407e4768be722a5787dea2c2 (127.30.8.1:42547): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 49f7b41be9cf45a0baeebcc337ec4cb8: Network error: Client connection negotiation failed: client connection to 127.30.8.1:42547: connect: Connection refused (error 111)
I20260811 15:08:41.892200 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 6553
I20260811 15:08:41.905157 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 5986
2026-08-11T15:08:41Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1125 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 15:08:41.936225 30752 mini_oidc.cc:152] Starting JWKS server
I20260811 15:08:41.937983 30752 webserver.cc:460] Webserver started at https://127.0.0.1:40277/ using document root <none> and password file <none>
I20260811 15:08:41.938131 30752 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 15:08:41.938374 30752 webserver.cc:460] Webserver started at http://127.0.0.1:45593/ using document root <none> and password file <none>
2026-08-11T15:08:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:08:41Z Disabled control of system clock
I20260811 15:08:41.948598 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:42585
--webserver_interface=127.30.8.62
--webserver_port=0
--builtin_ntp_servers=127.30.8.20:38099
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:42585
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40277/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786460893564915-30752-0/testchainca.cert with env {}
W20260811 15:08:42.064456 6844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:42.064759 6844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:42.064821 6844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:42.068970 6844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:08:42.069054 6844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:42.069077 6844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:42.069097 6844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:08:42.069118 6844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 15:08:42.069156 6844 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 15:08:42.069185 6844 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40277/jwks.json
I20260811 15:08:42.074808 6844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:38099
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:42585
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:42585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40277/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786460893564915-30752-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:42.076017 6844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:42.077057 6844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:42.085824 6852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:42.085843 6855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:42.085842 6844 server_base.cc:1034] running on GCE node
W20260811 15:08:42.085834 6851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:42.086642 6844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:42.087859 6844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:42.089030 6844 hybrid_clock.cc:648] HybridClock initialized: now 1786460922089001 us; error 44 us; skew 500 ppm
I20260811 15:08:42.091035 6844 webserver.cc:460] Webserver started at http://127.30.8.62:41787/ using document root <none> and password file <none>
I20260811 15:08:42.091636 6844 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:42.091713 6844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:42.091917 6844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:42.094022 6844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "61fe794441754c849245fbe857b2d6a6"
format_stamp: "Formatted at 2026-08-11 15:08:42 on dist-test-slave-1vmg"
server_key: "ab9bbd176bdae5f4f62ee4875eb2ca24"
server_key_iv: "e701f9bc26cc333428ede2b1de03a75c"
server_key_version: "encryptionkey@0"
I20260811 15:08:42.094588 6844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "61fe794441754c849245fbe857b2d6a6"
format_stamp: "Formatted at 2026-08-11 15:08:42 on dist-test-slave-1vmg"
server_key: "ab9bbd176bdae5f4f62ee4875eb2ca24"
server_key_iv: "e701f9bc26cc333428ede2b1de03a75c"
server_key_version: "encryptionkey@0"
I20260811 15:08:42.098369 6844 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.003s
I20260811 15:08:42.100927 6862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:42.102174 6844 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 15:08:42.102300 6844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "61fe794441754c849245fbe857b2d6a6"
format_stamp: "Formatted at 2026-08-11 15:08:42 on dist-test-slave-1vmg"
server_key: "ab9bbd176bdae5f4f62ee4875eb2ca24"
server_key_iv: "e701f9bc26cc333428ede2b1de03a75c"
server_key_version: "encryptionkey@0"
I20260811 15:08:42.102411 6844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:42.122012 6844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:42.122828 6844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:42.123049 6844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:42.130101 6844 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:42585
I20260811 15:08:42.130178 6950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:42585 every 8 connection(s)
I20260811 15:08:42.131197 6844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:08:42.134428 6951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:42.135032 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 6844
I20260811 15:08:42.135159 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:08:42.135466 30752 external_mini_cluster.cc:1428] Setting key 81b1973d41f0cfdedc04cead7498e00e
I20260811 15:08:42.141048 6951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6: Bootstrap starting.
I20260811 15:08:42.143373 6951 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:42.144222 6951 log.cc:824] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:42.144831 6954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:42.136940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39612 (local address 127.30.8.62:42585)
0811 15:08:42.137610 (+ 670us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:42.137625 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:42.137663 (+ 38us) server_negotiation.cc:407] Connection header received
0811 15:08:42.138322 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:42.138339 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:42.138758 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:42.139104 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:42.139539 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:42.140531 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:42.141450 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:42.141922 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:42.142066 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:42.142896 (+ 830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:42.142901 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:42.142950 (+ 49us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:08:42.143365 (+ 415us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:42.143420 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:08:42.143442 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:42.143528 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:42.143722 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 15:08:42.144001 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":317,"thread_start_us":140,"threads_started":1}
I20260811 15:08:42.146485 6951 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6: No bootstrap required, opened a new log
I20260811 15:08:42.147782 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:08:42.149832 6951 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fe794441754c849245fbe857b2d6a6" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42585 } }
I20260811 15:08:42.150110 6951 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:42.150173 6951 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61fe794441754c849245fbe857b2d6a6, State: Initialized, Role: FOLLOWER
I20260811 15:08:42.150588 6951 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fe794441754c849245fbe857b2d6a6" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42585 } }
I20260811 15:08:42.150715 6951 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:08:42.150781 6951 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:08:42.150882 6951 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:42.151854 6951 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fe794441754c849245fbe857b2d6a6" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42585 } }
I20260811 15:08:42.152194 6951 leader_election.cc:302] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61fe794441754c849245fbe857b2d6a6; no voters:
I20260811 15:08:42.152504 6951 leader_election.cc:288] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:08:42.152634 6958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:42.152877 6958 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6 [term 1 LEADER]: Becoming Leader. State: Replica: 61fe794441754c849245fbe857b2d6a6, State: Running, Role: LEADER
I20260811 15:08:42.153192 6958 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fe794441754c849245fbe857b2d6a6" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42585 } }
I20260811 15:08:42.153728 6951 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:08:42.155004 6960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61fe794441754c849245fbe857b2d6a6. Latest consensus state: current_term: 1 leader_uuid: "61fe794441754c849245fbe857b2d6a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fe794441754c849245fbe857b2d6a6" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42585 } } }
I20260811 15:08:42.155004 6959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61fe794441754c849245fbe857b2d6a6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61fe794441754c849245fbe857b2d6a6" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42585 } } }
I20260811 15:08:42.155189 6959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:42.155187 6960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:42.156244 6970 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:08:42.158892 6970 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:08:42.164461 6970 catalog_manager.cc:1347] Generated new cluster ID: 01b928d6772140a2be248e07d5bf62e9
I20260811 15:08:42.164546 6970 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:08:42.175398 6970 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:08:42.176537 6970 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:08:42.183866 6970 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61fe794441754c849245fbe857b2d6a6: Generated new TSK 0
I20260811 15:08:42.184777 6970 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 15:08:42.223431 6989 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 15:08:42.223610 6989 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 15:08:42.224155 6954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:42.204623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39624 (local address 127.30.8.62:42585)
0811 15:08:42.204768 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:42.204773 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:42.204923 (+ 150us) server_negotiation.cc:407] Connection header received
0811 15:08:42.205159 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:42.205162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:42.205215 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:42.205303 (+ 88us) server_negotiation.cc:226] Negotiated authn=JWT
0811 15:08:42.205596 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:42.206206 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:42.207495 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:42.207726 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:42.207804 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:42.208189 (+ 385us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 15:08:42.223665 (+ 15476us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40277/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 15:08:42.223986 (+ 321us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39624: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40277/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 15:08:42.224249 6954 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39624: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40277/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 15:08:42.225521 6988 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:08:42.204466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:42585 (local address 127.0.0.1:39624)
0811 15:08:42.204833 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:42.204849 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:42.204992 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:42.205337 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:42.205340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:42.205354 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0811 15:08:42.205481 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:42.205487 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:42.206312 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:42.206316 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:42.207383 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:42.207395 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:42.207817 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:42.207839 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:42.207933 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:42.207992 (+ 59us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 15:08:42.223917 (+ 15925us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40277/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 15:08:42.224172 (+ 255us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.8.62:42585: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40277/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":123,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.30.8.62:42585: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40277/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 15:08:42.226290 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 6844
2026-08-11T15:08:42Z chronyd exiting
I20260811 15:08:42.272765 30752 test_util.cc:175] -----------------------------------------------
I20260811 15:08:42.272861 30752 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786460893564915-30752-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (338 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 15:08:42.274026 30752 mini_oidc.cc:152] Starting JWKS server
I20260811 15:08:42.274900 30752 webserver.cc:460] Webserver started at https://127.0.0.1:35981/ using document root <none> and password file <none>
I20260811 15:08:42.275032 30752 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 15:08:42.275302 30752 webserver.cc:460] Webserver started at http://127.0.0.1:45723/ using document root <none> and password file <none>
2026-08-11T15:08:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:08:42Z Disabled control of system clock
I20260811 15:08:42.285326 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:37079
--webserver_interface=127.30.8.62
--webserver_port=0
--builtin_ntp_servers=127.30.8.20:37889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:37079
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35981/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786460893564915-30752-0/testchainca.cert with env {}
W20260811 15:08:42.400059 6997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:42.400370 6997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:42.400425 6997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:42.404513 6997 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:08:42.404604 6997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:42.404624 6997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:42.404646 6997 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:08:42.404666 6997 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 15:08:42.404688 6997 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 15:08:42.404716 6997 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35981/jwks.json
I20260811 15:08:42.410449 6997 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:37889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:37079
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:37079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35981/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786460893564915-30752-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:42.411688 6997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:42.412742 6997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:42.421391 7007 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:42.421391 7004 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:42.421437 7005 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:42.421741 6997 server_base.cc:1034] running on GCE node
I20260811 15:08:42.422354 6997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:42.423602 6997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:42.424767 6997 hybrid_clock.cc:648] HybridClock initialized: now 1786460922424749 us; error 33 us; skew 500 ppm
I20260811 15:08:42.426676 6997 webserver.cc:460] Webserver started at http://127.30.8.62:36749/ using document root <none> and password file <none>
I20260811 15:08:42.427317 6997 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:42.427385 6997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:42.427600 6997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:42.429692 6997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "46748bb798944bac8ad1b0defd218965"
format_stamp: "Formatted at 2026-08-11 15:08:42 on dist-test-slave-1vmg"
server_key: "fc00e57a7f14878293e042d722b33db9"
server_key_iv: "51a214ceaca8e2fc2dfbb249d44513cf"
server_key_version: "encryptionkey@0"
I20260811 15:08:42.430261 6997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "46748bb798944bac8ad1b0defd218965"
format_stamp: "Formatted at 2026-08-11 15:08:42 on dist-test-slave-1vmg"
server_key: "fc00e57a7f14878293e042d722b33db9"
server_key_iv: "51a214ceaca8e2fc2dfbb249d44513cf"
server_key_version: "encryptionkey@0"
I20260811 15:08:42.434056 6997 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 15:08:42.436656 7018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:42.437929 6997 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.001s
I20260811 15:08:42.438047 6997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "46748bb798944bac8ad1b0defd218965"
format_stamp: "Formatted at 2026-08-11 15:08:42 on dist-test-slave-1vmg"
server_key: "fc00e57a7f14878293e042d722b33db9"
server_key_iv: "51a214ceaca8e2fc2dfbb249d44513cf"
server_key_version: "encryptionkey@0"
I20260811 15:08:42.438175 6997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:42.452906 6997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:42.453720 6997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:42.453913 6997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:42.460865 6997 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:37079
I20260811 15:08:42.460902 7107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:37079 every 8 connection(s)
I20260811 15:08:42.461871 6997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:08:42.465296 7109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:42.471262 7109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965: Bootstrap starting.
I20260811 15:08:42.471499 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 6997
I20260811 15:08:42.471623 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:08:42.471909 30752 external_mini_cluster.cc:1428] Setting key d62acf50553eada8b9ca68fd08991793
I20260811 15:08:42.474040 7109 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:42.474833 7109 log.cc:824] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:42.477160 7109 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965: No bootstrap required, opened a new log
I20260811 15:08:42.481554 7109 raft_consensus.cc:348] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46748bb798944bac8ad1b0defd218965" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 37079 } }
I20260811 15:08:42.481854 7109 raft_consensus.cc:374] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:42.481913 7109 raft_consensus.cc:729] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 46748bb798944bac8ad1b0defd218965, State: Initialized, Role: FOLLOWER
I20260811 15:08:42.482261 7112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:42.473269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46368 (local address 127.30.8.62:37079)
0811 15:08:42.473936 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:42.473952 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:42.473989 (+ 37us) server_negotiation.cc:407] Connection header received
0811 15:08:42.474701 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:42.474720 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:42.475242 (+ 522us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:42.475540 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:42.476012 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:42.477411 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:42.478304 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:42.479076 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:42.479244 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:42.480029 (+ 785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:42.480036 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:42.480055 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:08:42.480621 (+ 566us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:42.480714 (+ 93us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:08:42.480743 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:42.480840 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:42.480930 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 15:08:42.481224 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":133,"threads_started":1}
I20260811 15:08:42.482419 7109 consensus_queue.cc:260] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46748bb798944bac8ad1b0defd218965" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 37079 } }
I20260811 15:08:42.482578 7109 raft_consensus.cc:388] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:08:42.482649 7109 raft_consensus.cc:482] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:08:42.482749 7109 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:42.483739 7109 raft_consensus.cc:504] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46748bb798944bac8ad1b0defd218965" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 37079 } }
I20260811 15:08:42.484073 7109 leader_election.cc:302] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 46748bb798944bac8ad1b0defd218965; no voters:
I20260811 15:08:42.484457 7109 leader_election.cc:288] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:08:42.484603 7115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:42.485200 7115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965 [term 1 LEADER]: Becoming Leader. State: Replica: 46748bb798944bac8ad1b0defd218965, State: Running, Role: LEADER
I20260811 15:08:42.485339 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:08:42.485586 7115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46748bb798944bac8ad1b0defd218965" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 37079 } }
I20260811 15:08:42.485764 7109 sys_catalog.cc:564] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:08:42.487186 7120 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 46748bb798944bac8ad1b0defd218965. Latest consensus state: current_term: 1 leader_uuid: "46748bb798944bac8ad1b0defd218965" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46748bb798944bac8ad1b0defd218965" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 37079 } } }
I20260811 15:08:42.487314 7120 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:42.487622 7117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "46748bb798944bac8ad1b0defd218965" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "46748bb798944bac8ad1b0defd218965" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 37079 } } }
I20260811 15:08:42.487703 7117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:42.488013 7133 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:08:42.491098 7133 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:08:42.496759 7133 catalog_manager.cc:1347] Generated new cluster ID: 67263c34464744d481cbf1e3dede7ef4
I20260811 15:08:42.496845 7133 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:08:42.519342 7133 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:08:42.520355 7133 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:08:42.527675 7133 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 46748bb798944bac8ad1b0defd218965: Generated new TSK 0
I20260811 15:08:42.528407 7133 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 15:08:42.553411 7158 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 15:08:42.553506 7158 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 15:08:42.554399 7112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:42.538089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46372 (local address 127.30.8.62:37079)
0811 15:08:42.538225 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:42.538231 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:42.538339 (+ 108us) server_negotiation.cc:407] Connection header received
0811 15:08:42.538502 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:42.538505 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:42.538576 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:42.538705 (+ 129us) server_negotiation.cc:226] Negotiated authn=JWT
0811 15:08:42.538891 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:42.539798 (+ 907us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:42.540676 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:42.541025 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:42.541124 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:42.541416 (+ 292us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 15:08:42.553924 (+ 12508us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35981/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 15:08:42.554244 (+ 320us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46372: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35981/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 15:08:42.554479 7112 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46372: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35981/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260811 15:08:42.554508 7157 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:08:42.537976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:37079 (local address 127.0.0.1:46372)
0811 15:08:42.538251 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:42.538264 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:42.538364 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:42.538693 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:42.538695 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:42.538701 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0811 15:08:42.538798 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:42.538803 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:42.539935 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:42.539937 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:42.540567 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:42.540574 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:42.541117 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:42.541134 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:42.541199 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:42.541222 (+ 23us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 15:08:42.554177 (+ 12955us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35981/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 15:08:42.554349 (+ 172us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.8.62:37079: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35981/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":93,"threads_started":1}
I20260811 15:08:42.555248 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 6997
2026-08-11T15:08:42Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (331 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 15:08:42.605850 30752 mini_oidc.cc:152] Starting JWKS server
I20260811 15:08:42.606748 30752 webserver.cc:460] Webserver started at https://127.0.0.1:33033/ using document root <none> and password file <none>
I20260811 15:08:42.606848 30752 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 15:08:42.607138 30752 webserver.cc:460] Webserver started at http://127.0.0.1:44611/ using document root <none> and password file <none>
2026-08-11T15:08:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:08:42Z Disabled control of system clock
I20260811 15:08:42.617158 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:39471
--webserver_interface=127.30.8.62
--webserver_port=0
--builtin_ntp_servers=127.30.8.20:32815
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:39471
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33033/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 15:08:42.732771 7167 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:42.733074 7167 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:42.733139 7167 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:42.737201 7167 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:08:42.737286 7167 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:42.737306 7167 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:42.737329 7167 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:08:42.737349 7167 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 15:08:42.737368 7167 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 15:08:42.737389 7167 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33033/jwks.json
I20260811 15:08:42.742352 7167 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:32815
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:39471
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:39471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33033/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7167
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:42.743600 7167 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:42.745038 7167 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:42.753542 7175 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:42.753582 7179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:42.753590 7184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:42.753865 7167 server_base.cc:1034] running on GCE node
I20260811 15:08:42.754457 7167 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:42.755667 7167 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:42.756829 7167 hybrid_clock.cc:648] HybridClock initialized: now 1786460922756808 us; error 36 us; skew 500 ppm
I20260811 15:08:42.758759 7167 webserver.cc:460] Webserver started at http://127.30.8.62:39331/ using document root <none> and password file <none>
I20260811 15:08:42.759402 7167 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:42.759469 7167 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:42.759689 7167 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:42.761842 7167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "2029b625a32c4bd4b47713f5345eb397"
format_stamp: "Formatted at 2026-08-11 15:08:42 on dist-test-slave-1vmg"
server_key: "c4db9127f4be6155c2fe09525d77e24e"
server_key_iv: "e8c28871d1d85f1549539971e3d25705"
server_key_version: "encryptionkey@0"
I20260811 15:08:42.762403 7167 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "2029b625a32c4bd4b47713f5345eb397"
format_stamp: "Formatted at 2026-08-11 15:08:42 on dist-test-slave-1vmg"
server_key: "c4db9127f4be6155c2fe09525d77e24e"
server_key_iv: "e8c28871d1d85f1549539971e3d25705"
server_key_version: "encryptionkey@0"
I20260811 15:08:42.766173 7167 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 15:08:42.768672 7194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:42.769920 7167 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 15:08:42.770047 7167 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "2029b625a32c4bd4b47713f5345eb397"
format_stamp: "Formatted at 2026-08-11 15:08:42 on dist-test-slave-1vmg"
server_key: "c4db9127f4be6155c2fe09525d77e24e"
server_key_iv: "e8c28871d1d85f1549539971e3d25705"
server_key_version: "encryptionkey@0"
I20260811 15:08:42.770166 7167 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:42.796077 7167 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:42.796856 7167 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:42.797044 7167 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:42.804157 7167 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:39471
I20260811 15:08:42.804186 7284 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:39471 every 8 connection(s)
I20260811 15:08:42.805171 7167 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:08:42.808581 7285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:42.813378 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 7167
I20260811 15:08:42.813509 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:08:42.813798 30752 external_mini_cluster.cc:1428] Setting key eef1bb0dde944b7fe8d42378775dc864
I20260811 15:08:42.814646 7285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397: Bootstrap starting.
I20260811 15:08:42.817103 7285 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:42.817976 7285 log.cc:824] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:42.820248 7285 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397: No bootstrap required, opened a new log
I20260811 15:08:42.823611 7289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:42.815292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53884 (local address 127.30.8.62:39471)
0811 15:08:42.815958 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:42.815973 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:42.816009 (+ 36us) server_negotiation.cc:407] Connection header received
0811 15:08:42.816662 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:42.816679 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:42.817074 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:42.817398 (+ 324us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:42.817767 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:42.819000 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:42.820161 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:42.820650 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:42.820768 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:42.821761 (+ 993us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:42.821767 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:42.821781 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:08:42.822190 (+ 409us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:42.822251 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:08:42.822271 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:42.822347 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:42.822528 (+ 181us) server_negotiation.cc:299] Negotiation successful
0811 15:08:42.822803 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":137,"threads_started":1}
I20260811 15:08:42.824473 7285 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2029b625a32c4bd4b47713f5345eb397" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 39471 } }
I20260811 15:08:42.824781 7285 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:42.824851 7285 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2029b625a32c4bd4b47713f5345eb397, State: Initialized, Role: FOLLOWER
I20260811 15:08:42.825345 7285 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2029b625a32c4bd4b47713f5345eb397" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 39471 } }
I20260811 15:08:42.825515 7285 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:08:42.825589 7285 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:08:42.825700 7285 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:42.826596 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:08:42.826990 7285 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2029b625a32c4bd4b47713f5345eb397" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 39471 } }
I20260811 15:08:42.827348 7285 leader_election.cc:302] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2029b625a32c4bd4b47713f5345eb397; no voters:
I20260811 15:08:42.827754 7285 leader_election.cc:288] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:08:42.827867 7291 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:42.828074 7291 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397 [term 1 LEADER]: Becoming Leader. State: Replica: 2029b625a32c4bd4b47713f5345eb397, State: Running, Role: LEADER
I20260811 15:08:42.828399 7291 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2029b625a32c4bd4b47713f5345eb397" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 39471 } }
I20260811 15:08:42.829159 7285 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:08:42.830235 7295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2029b625a32c4bd4b47713f5345eb397" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2029b625a32c4bd4b47713f5345eb397" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 39471 } } }
I20260811 15:08:42.830368 7295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:42.830617 7296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2029b625a32c4bd4b47713f5345eb397. Latest consensus state: current_term: 1 leader_uuid: "2029b625a32c4bd4b47713f5345eb397" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2029b625a32c4bd4b47713f5345eb397" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 39471 } } }
I20260811 15:08:42.830708 7296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:42.830904 7307 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:08:42.833762 7307 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:08:42.839768 7307 catalog_manager.cc:1347] Generated new cluster ID: f39715cf0e364ca6b28778ee7d4e5a99
I20260811 15:08:42.839865 7307 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:08:42.867242 7307 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:08:42.868630 7307 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:08:42.874317 7307 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2029b625a32c4bd4b47713f5345eb397: Generated new TSK 0
I20260811 15:08:42.875222 7307 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:08:42.895895 7289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:42.890411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53888 (local address 127.30.8.62:39471)
0811 15:08:42.890579 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:42.890584 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:42.890744 (+ 160us) server_negotiation.cc:407] Connection header received
0811 15:08:42.890938 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:42.890942 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:42.891009 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:42.891145 (+ 136us) server_negotiation.cc:226] Negotiated authn=JWT
0811 15:08:42.891385 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:42.892243 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:42.893099 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:42.893460 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:42.893561 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:42.893833 (+ 272us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 15:08:42.895476 (+ 1643us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33033/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 15:08:42.895729 (+ 253us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53888: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33033/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 15:08:42.895978 7289 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53888: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33033/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 15:08:42.897397 7338 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:08:42.890331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:39471 (local address 127.0.0.1:53888)
0811 15:08:42.890645 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:42.890659 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:42.890766 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:42.891164 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:42.891167 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:42.891175 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 15:08:42.891282 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:42.891287 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:42.892347 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:42.892349 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:42.892986 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:42.892993 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:42.893556 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:42.893576 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:42.893633 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:42.893657 (+ 24us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 15:08:42.895654 (+ 1997us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33033/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 15:08:42.897230 (+ 1576us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.8.62:39471: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33033/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":233,"mutex_wait_us":1380,"thread_start_us":129,"threads_started":1}
I20260811 15:08:42.898238 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 7167
E20260811 15:08:42.909358 7339 webserver.cc:533] Webserver: sslize failed:
E20260811 15:08:42.909451 7339 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T15:08:42Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (341 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T15:08:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:08:42Z Disabled control of system clock
I20260811 15:08:42.956650 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:34087
--webserver_interface=127.30.8.62
--webserver_port=0
--builtin_ntp_servers=127.30.8.20:41521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:34087
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786460893564915-30752-0/testchainca.cert with env {}
W20260811 15:08:43.071851 7348 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:43.072152 7348 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:43.072192 7348 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:43.076195 7348 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:08:43.076270 7348 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:43.076284 7348 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:43.076301 7348 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:08:43.076315 7348 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 15:08:43.076331 7348 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 15:08:43.076344 7348 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 15:08:43.082082 7348 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:41521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:34087
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:34087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786460893564915-30752-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7348
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:43.083298 7348 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:43.084676 7348 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:43.097532 7355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:43.097560 7354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:43.097560 7360 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:43.097781 7348 server_base.cc:1034] running on GCE node
I20260811 15:08:43.098465 7348 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:43.099674 7348 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:43.100858 7348 hybrid_clock.cc:648] HybridClock initialized: now 1786460923100840 us; error 40 us; skew 500 ppm
I20260811 15:08:43.102847 7348 webserver.cc:460] Webserver started at http://127.30.8.62:35197/ using document root <none> and password file <none>
I20260811 15:08:43.103483 7348 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:43.103551 7348 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:43.103736 7348 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:43.105835 7348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "a56bf3df84154786927b67bd0afd3c64"
format_stamp: "Formatted at 2026-08-11 15:08:43 on dist-test-slave-1vmg"
server_key: "1ae6d93e2d89b9377ff9a98b1fb53f2f"
server_key_iv: "025c647801a48203695875703cd3ec09"
server_key_version: "encryptionkey@0"
I20260811 15:08:43.106386 7348 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "a56bf3df84154786927b67bd0afd3c64"
format_stamp: "Formatted at 2026-08-11 15:08:43 on dist-test-slave-1vmg"
server_key: "1ae6d93e2d89b9377ff9a98b1fb53f2f"
server_key_iv: "025c647801a48203695875703cd3ec09"
server_key_version: "encryptionkey@0"
I20260811 15:08:43.110173 7348 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 15:08:43.112764 7366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:43.114017 7348 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 15:08:43.114136 7348 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "a56bf3df84154786927b67bd0afd3c64"
format_stamp: "Formatted at 2026-08-11 15:08:43 on dist-test-slave-1vmg"
server_key: "1ae6d93e2d89b9377ff9a98b1fb53f2f"
server_key_iv: "025c647801a48203695875703cd3ec09"
server_key_version: "encryptionkey@0"
I20260811 15:08:43.114264 7348 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:43.140144 7348 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:43.140933 7348 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:43.141127 7348 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:43.148345 7348 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:34087
I20260811 15:08:43.148361 7444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:34087 every 8 connection(s)
I20260811 15:08:43.149382 7348 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:08:43.152453 7445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:43.153736 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 7348
I20260811 15:08:43.153859 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:08:43.154153 30752 external_mini_cluster.cc:1428] Setting key 30ccf31407a3931d55d383a1359f1505
I20260811 15:08:43.158804 7445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64: Bootstrap starting.
I20260811 15:08:43.161562 7445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:43.162331 7445 log.cc:824] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:43.163403 7455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:43.155555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57366 (local address 127.30.8.62:34087)
0811 15:08:43.156208 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:43.156232 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:43.156272 (+ 40us) server_negotiation.cc:407] Connection header received
0811 15:08:43.156897 (+ 625us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:43.156914 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:43.157315 (+ 401us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:43.157594 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:43.157954 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:43.158901 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:43.160101 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:43.160581 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:43.160697 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:43.161589 (+ 892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:43.161594 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:43.161608 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:08:43.162040 (+ 432us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:43.162096 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:08:43.162118 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:43.162194 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:43.162356 (+ 162us) server_negotiation.cc:299] Negotiation successful
0811 15:08:43.162616 (+ 260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":120,"threads_started":1}
I20260811 15:08:43.164405 7445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64: No bootstrap required, opened a new log
I20260811 15:08:43.166218 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:08:43.168196 7445 raft_consensus.cc:348] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a56bf3df84154786927b67bd0afd3c64" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34087 } }
I20260811 15:08:43.168423 7445 raft_consensus.cc:374] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:43.168469 7445 raft_consensus.cc:729] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a56bf3df84154786927b67bd0afd3c64, State: Initialized, Role: FOLLOWER
I20260811 15:08:43.168867 7445 consensus_queue.cc:260] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a56bf3df84154786927b67bd0afd3c64" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34087 } }
I20260811 15:08:43.168989 7445 raft_consensus.cc:388] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:08:43.169052 7445 raft_consensus.cc:482] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:08:43.169143 7445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:43.170086 7445 raft_consensus.cc:504] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a56bf3df84154786927b67bd0afd3c64" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34087 } }
I20260811 15:08:43.170420 7445 leader_election.cc:302] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a56bf3df84154786927b67bd0afd3c64; no voters:
I20260811 15:08:43.170737 7445 leader_election.cc:288] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:08:43.170850 7461 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:43.171092 7461 raft_consensus.cc:686] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64 [term 1 LEADER]: Becoming Leader. State: Replica: a56bf3df84154786927b67bd0afd3c64, State: Running, Role: LEADER
I20260811 15:08:43.171422 7461 consensus_queue.cc:237] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a56bf3df84154786927b67bd0afd3c64" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34087 } }
I20260811 15:08:43.172103 7445 sys_catalog.cc:564] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:08:43.173179 7463 sys_catalog.cc:455] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a56bf3df84154786927b67bd0afd3c64. Latest consensus state: current_term: 1 leader_uuid: "a56bf3df84154786927b67bd0afd3c64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a56bf3df84154786927b67bd0afd3c64" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34087 } } }
I20260811 15:08:43.173314 7463 sys_catalog.cc:458] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:43.173550 7462 sys_catalog.cc:455] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a56bf3df84154786927b67bd0afd3c64" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a56bf3df84154786927b67bd0afd3c64" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34087 } } }
I20260811 15:08:43.173646 7462 sys_catalog.cc:458] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:43.173818 7474 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:08:43.177099 7474 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:08:43.182901 7474 catalog_manager.cc:1347] Generated new cluster ID: e37bc1c10be340ec971a938c2c7be53f
I20260811 15:08:43.183017 7474 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:08:43.199113 7474 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:08:43.200192 7474 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:08:43.205209 7474 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a56bf3df84154786927b67bd0afd3c64: Generated new TSK 0
I20260811 15:08:43.205925 7474 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:08:43.295998 7455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:43.215098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57368 (local address 127.30.8.62:34087)
0811 15:08:43.215234 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:43.215239 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:43.215689 (+ 450us) server_negotiation.cc:407] Connection header received
0811 15:08:43.215939 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:43.215943 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:43.216006 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:43.216127 (+ 121us) server_negotiation.cc:226] Negotiated authn=JWT
0811 15:08:43.216381 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:43.216943 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:43.217866 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:43.218158 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:43.218283 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:43.218641 (+ 358us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 15:08:43.295584 (+ 76943us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 15:08:43.295875 (+ 291us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57368: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 15:08:43.296079 7455 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57368: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 15:08:43.296303 7500 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:08:43.215027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34087 (local address 127.0.0.1:57368)
0811 15:08:43.215594 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:43.215609 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:43.215734 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:43.216134 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:43.216138 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:43.216146 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 15:08:43.216271 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:43.216277 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:43.217061 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:43.217065 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:43.217743 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:43.217752 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:43.218303 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:43.218327 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:43.218429 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:43.218463 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 15:08:43.295868 (+ 77405us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 15:08:43.296112 (+ 244us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.8.62:34087: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":494,"thread_start_us":231,"threads_started":1}
I20260811 15:08:43.297053 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 7348
2026-08-11T15:08:43Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (387 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (129 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (129 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:08:43 dist-test-slave-1vmg krb5kdc[7516](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:08:43 dist-test-slave-1vmg krb5kdc[7516](info): set up 2 sockets
Aug 11 15:08:43 dist-test-slave-1vmg krb5kdc[7516](info): commencing operation
krb5kdc: starting...
W20260811 15:08:44.765287 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:08:44 dist-test-slave-1vmg krb5kdc[7516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:08:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:08:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:08:44.895378 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:33265
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:33579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:33265
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:45.012092 7537 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:45.012393 7537 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:45.012455 7537 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:45.016568 7537 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:08:45.016647 7537 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:45.016676 7537 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:45.016708 7537 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:08:45.016739 7537 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:08:45.022336 7537 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:33579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:33265
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:33265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7537
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:45.023548 7537 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:45.024626 7537 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:45.033143 7543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:45.033145 7542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:45.033215 7537 server_base.cc:1034] running on GCE node
W20260811 15:08:45.033145 7546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:45.033983 7537 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:45.035249 7537 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:45.036429 7537 hybrid_clock.cc:648] HybridClock initialized: now 1786460925036411 us; error 59 us; skew 500 ppm
Aug 11 15:08:45 dist-test-slave-1vmg krb5kdc[7516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460925, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:45.039286 7537 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:08:45.040288 7537 webserver.cc:460] Webserver started at http://127.30.8.62:40431/ using document root <none> and password file <none>
I20260811 15:08:45.040884 7537 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:45.040957 7537 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:45.041160 7537 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:45.043354 7537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "8f7e50cda36440dd9a73b0c76c9e5abe"
format_stamp: "Formatted at 2026-08-11 15:08:45 on dist-test-slave-1vmg"
server_key: "3134d5bc55e450703499642aaba3d2f9"
server_key_iv: "3576cec330530ac02b5c20e41fa6015d"
server_key_version: "encryptionkey@0"
I20260811 15:08:45.043910 7537 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "8f7e50cda36440dd9a73b0c76c9e5abe"
format_stamp: "Formatted at 2026-08-11 15:08:45 on dist-test-slave-1vmg"
server_key: "3134d5bc55e450703499642aaba3d2f9"
server_key_iv: "3576cec330530ac02b5c20e41fa6015d"
server_key_version: "encryptionkey@0"
I20260811 15:08:45.047750 7537 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 15:08:45.050164 7559 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:45.051564 7537 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:08:45.051714 7537 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "8f7e50cda36440dd9a73b0c76c9e5abe"
format_stamp: "Formatted at 2026-08-11 15:08:45 on dist-test-slave-1vmg"
server_key: "3134d5bc55e450703499642aaba3d2f9"
server_key_iv: "3576cec330530ac02b5c20e41fa6015d"
server_key_version: "encryptionkey@0"
I20260811 15:08:45.051843 7537 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:45.072660 7537 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:45.075855 7537 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:45.076033 7537 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:45.084837 7537 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:33265
I20260811 15:08:45.084928 7657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:33265 every 8 connection(s)
I20260811 15:08:45.086371 7537 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:08:45.090327 7658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:45.092782 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 7537
I20260811 15:08:45.092938 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:08:45.093281 30752 external_mini_cluster.cc:1428] Setting key 1b1eff967fce7a5a1eb34e008189f8d3
I20260811 15:08:45.097311 7658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe: Bootstrap starting.
I20260811 15:08:45.100409 7658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:45.101275 7658 log.cc:824] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:45.103477 7658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe: No bootstrap required, opened a new log
Aug 11 15:08:45 dist-test-slave-1vmg krb5kdc[7516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:45.107587 7658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7e50cda36440dd9a73b0c76c9e5abe" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 33265 } }
I20260811 15:08:45.107854 7658 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:45.107925 7658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f7e50cda36440dd9a73b0c76c9e5abe, State: Initialized, Role: FOLLOWER
I20260811 15:08:45.108435 7658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7e50cda36440dd9a73b0c76c9e5abe" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 33265 } }
I20260811 15:08:45.108569 7658 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:08:45.108631 7658 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:08:45.108738 7658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:45.109752 7658 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7e50cda36440dd9a73b0c76c9e5abe" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 33265 } }
I20260811 15:08:45.110095 7658 leader_election.cc:302] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f7e50cda36440dd9a73b0c76c9e5abe; no voters:
I20260811 15:08:45.110406 7658 leader_election.cc:288] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:08:45.110505 7666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:45.110607 7663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:45.095535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53740 (local address 127.30.8.62:33265)
0811 15:08:45.096173 (+ 638us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:45.096211 (+ 38us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:45.096267 (+ 56us) server_negotiation.cc:407] Connection header received
0811 15:08:45.097133 (+ 866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:45.097160 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:45.097637 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:45.098041 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:45.098842 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:45.100389 (+ 1547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:45.101323 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:45.101941 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:45.102076 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:45.104812 (+ 2736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:45.104820 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:45.104839 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:45.104873 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:45.107400 (+ 2527us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:45.107857 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:45.107864 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:45.107870 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:45.108027 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:45.108318 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:45.108322 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:45.108324 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:45.108863 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:45.109080 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:45.109383 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 15:08:45.109692 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":430,"thread_start_us":187,"threads_started":1}
I20260811 15:08:45.110677 7666 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe [term 1 LEADER]: Becoming Leader. State: Replica: 8f7e50cda36440dd9a73b0c76c9e5abe, State: Running, Role: LEADER
I20260811 15:08:45.111003 7666 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7e50cda36440dd9a73b0c76c9e5abe" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 33265 } }
I20260811 15:08:45.111781 7658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:08:45.112296 7668 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f7e50cda36440dd9a73b0c76c9e5abe. Latest consensus state: current_term: 1 leader_uuid: "8f7e50cda36440dd9a73b0c76c9e5abe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7e50cda36440dd9a73b0c76c9e5abe" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 33265 } } }
I20260811 15:08:45.112416 7668 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:45.112696 7667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f7e50cda36440dd9a73b0c76c9e5abe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f7e50cda36440dd9a73b0c76c9e5abe" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 33265 } } }
I20260811 15:08:45.112857 7667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:45.113170 7670 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:08:45.116972 7670 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:08:45.123039 7670 catalog_manager.cc:1347] Generated new cluster ID: fb8d228d873244128d9537e59eb3c892
I20260811 15:08:45.123128 7670 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:08:45.134732 7670 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:08:45.135883 7670 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:08:45.143829 7670 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f7e50cda36440dd9a73b0c76c9e5abe: Generated new TSK 0
I20260811 15:08:45.144692 7670 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
I20260811 15:08:45.190397 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:33265
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:33579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:45.339498 7696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:45.339850 7696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:45.339898 7696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:45.344165 7696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:45.344249 7696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:45.344312 7696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:08:45.350063 7696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:33579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:33265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:45.351363 7696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:45.352458 7696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:45.362874 7708 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:45.362959 7714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:45.362888 7709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:45.363166 7696 server_base.cc:1034] running on GCE node
I20260811 15:08:45.363664 7696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:45.364413 7696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:45.365587 7696 hybrid_clock.cc:648] HybridClock initialized: now 1786460925365574 us; error 29 us; skew 500 ppm
Aug 11 15:08:45 dist-test-slave-1vmg krb5kdc[7516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460925, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:45.368597 7696 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:08:45.369680 7696 webserver.cc:460] Webserver started at http://127.30.8.1:44343/ using document root <none> and password file <none>
I20260811 15:08:45.370303 7696 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:45.370373 7696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:45.370605 7696 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:45.372864 7696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "0b4d408091384400b90df4666ad567a5"
format_stamp: "Formatted at 2026-08-11 15:08:45 on dist-test-slave-1vmg"
server_key: "2dccdba7c2238e058cee36f1d45d641e"
server_key_iv: "ddf8e9ce289f1eb7a7d0e2196232065e"
server_key_version: "encryptionkey@0"
I20260811 15:08:45.373435 7696 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "0b4d408091384400b90df4666ad567a5"
format_stamp: "Formatted at 2026-08-11 15:08:45 on dist-test-slave-1vmg"
server_key: "2dccdba7c2238e058cee36f1d45d641e"
server_key_iv: "ddf8e9ce289f1eb7a7d0e2196232065e"
server_key_version: "encryptionkey@0"
I20260811 15:08:45.377919 7696 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 15:08:45.381659 7723 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:45.383477 7696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 15:08:45.383642 7696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "0b4d408091384400b90df4666ad567a5"
format_stamp: "Formatted at 2026-08-11 15:08:45 on dist-test-slave-1vmg"
server_key: "2dccdba7c2238e058cee36f1d45d641e"
server_key_iv: "ddf8e9ce289f1eb7a7d0e2196232065e"
server_key_version: "encryptionkey@0"
I20260811 15:08:45.383783 7696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:45.401047 7696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:45.405037 7696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:45.405315 7696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:45.406119 7696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:45.407305 7696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:45.407361 7696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:45.407426 7696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:45.407459 7696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:45.419126 7696 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:46755
I20260811 15:08:45.419234 7906 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:46755 every 8 connection(s)
I20260811 15:08:45.420389 7696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:08:45.430301 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 7696
I20260811 15:08:45.430444 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:08:45.431135 30752 external_mini_cluster.cc:1428] Setting key 07e6f18de809a42fa6c41cdbfe774e34
Aug 11 15:08:45 dist-test-slave-1vmg krb5kdc[7516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460925, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:45.436480 7663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:45.422725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:43067 (local address 127.30.8.62:33265)
0811 15:08:45.422863 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:45.422867 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:45.423903 (+ 1036us) server_negotiation.cc:407] Connection header received
0811 15:08:45.425378 (+ 1475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:45.425381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:45.425443 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:45.425519 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:45.426836 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:45.427434 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:45.428602 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:45.428829 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:45.428920 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:45.432135 (+ 3215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:45.432140 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:45.432143 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:45.432186 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:45.433785 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:45.434535 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:45.434539 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:45.434541 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:45.434661 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:45.435139 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:45.435143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:45.435145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:45.435433 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:45.435544 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:45.436114 (+ 570us) server_negotiation.cc:299] Negotiation successful
0811 15:08:45.436301 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 15:08:45.437284 7910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:45.423170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:33265 (local address 127.30.8.1:43067)
0811 15:08:45.423737 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:45.423784 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:45.425131 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:45.425742 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:45.425750 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:45.426138 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:45.426654 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:45.426668 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:45.427567 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:45.427570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:45.428467 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:45.428477 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:45.428989 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:45.428993 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:45.429165 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:45.429857 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:45.429880 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:45.431863 (+ 1983us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:45.433947 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:45.433954 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:45.433977 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:45.434376 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:45.434780 (+ 404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:45.434783 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:45.434785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:45.435025 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:45.435542 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:45.435548 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:45.435841 (+ 293us) client_negotiation.cc:769] Sending connection context
0811 15:08:45.436013 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 15:08:45.436318 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":185,"threads_started":1}
I20260811 15:08:45.438642 7908 heartbeater.cc:347] Connected to a master server at 127.30.8.62:33265
I20260811 15:08:45.438966 7908 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:45.439555 7908 heartbeater.cc:511] Master 127.30.8.62:33265 requested a full tablet report, sending...
I20260811 15:08:45.441264 7588 ts_manager.cc:194] Registered new tserver with Master: 0b4d408091384400b90df4666ad567a5 (127.30.8.1:46755)
WARNING: no policy specified for kudu/127.30.8.2@KRBTEST.COM; defaulting to no policy
I20260811 15:08:45.442490 7588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:43067
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
I20260811 15:08:45.462683 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:33265
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:33579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:45.583400 7916 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:45.583727 7916 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:45.583791 7916 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:45.588133 7916 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:45.588227 7916 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:45.588317 7916 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:08:45.594214 7916 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:33579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:33265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7916
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:45.595533 7916 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:45.596659 7916 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:45.607029 7935 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:45.607021 7929 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:45.607021 7933 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:45.607095 7916 server_base.cc:1034] running on GCE node
I20260811 15:08:45.607731 7916 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:45.608446 7916 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:45.609634 7916 hybrid_clock.cc:648] HybridClock initialized: now 1786460925609610 us; error 37 us; skew 500 ppm
Aug 11 15:08:45 dist-test-slave-1vmg krb5kdc[7516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460925, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:45.612982 7916 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:08:45.614476 7916 webserver.cc:460] Webserver started at http://127.30.8.2:35883/ using document root <none> and password file <none>
I20260811 15:08:45.615289 7916 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:45.615383 7916 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:45.615667 7916 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:45.618337 7916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "11a79800f2ea4f81a9217a4ae60f92bf"
format_stamp: "Formatted at 2026-08-11 15:08:45 on dist-test-slave-1vmg"
server_key: "f60e3997f93edb910b8ced0d2a13b6c9"
server_key_iv: "a8b1a2daddd57f5df1e59f6b0882f2b7"
server_key_version: "encryptionkey@0"
I20260811 15:08:45.619056 7916 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "11a79800f2ea4f81a9217a4ae60f92bf"
format_stamp: "Formatted at 2026-08-11 15:08:45 on dist-test-slave-1vmg"
server_key: "f60e3997f93edb910b8ced0d2a13b6c9"
server_key_iv: "a8b1a2daddd57f5df1e59f6b0882f2b7"
server_key_version: "encryptionkey@0"
I20260811 15:08:45.624125 7916 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 15:08:45.627504 7946 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:45.629288 7916 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 15:08:45.629464 7916 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "11a79800f2ea4f81a9217a4ae60f92bf"
format_stamp: "Formatted at 2026-08-11 15:08:45 on dist-test-slave-1vmg"
server_key: "f60e3997f93edb910b8ced0d2a13b6c9"
server_key_iv: "a8b1a2daddd57f5df1e59f6b0882f2b7"
server_key_version: "encryptionkey@0"
I20260811 15:08:45.629614 7916 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:45.651912 7916 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:45.656512 7916 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:45.656764 7916 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:45.657444 7916 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:45.658653 7916 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:45.658721 7916 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:45.658787 7916 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:45.658819 7916 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:45.673918 7916 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:41599
I20260811 15:08:45.673960 8134 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:41599 every 8 connection(s)
I20260811 15:08:45.675284 7916 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:08:45.684640 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 7916
I20260811 15:08:45.684810 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:08:45.685148 30752 external_mini_cluster.cc:1428] Setting key dc2413bdd314f1bb21a6c72700399ce3
Aug 11 15:08:45 dist-test-slave-1vmg krb5kdc[7516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460925, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:45.695905 7663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:45.681488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:49073 (local address 127.30.8.62:33265)
0811 15:08:45.681654 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:45.681660 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:45.681684 (+ 24us) server_negotiation.cc:407] Connection header received
0811 15:08:45.683312 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:45.683316 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:45.683401 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:45.683546 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:45.685536 (+ 1990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:45.686346 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:45.688013 (+ 1667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:45.688374 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:45.688499 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:45.691606 (+ 3107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:45.691610 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:45.691613 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:45.691643 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:45.693374 (+ 1731us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:45.694024 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:45.694028 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:45.694030 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:45.694112 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:45.694518 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:45.694521 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:45.694523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:45.694790 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:45.694880 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:45.695548 (+ 668us) server_negotiation.cc:299] Negotiation successful
0811 15:08:45.695727 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 15:08:45.696903 8137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:45.679994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:33265 (local address 127.30.8.2:49073)
0811 15:08:45.680907 (+ 913us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:45.681004 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:45.682973 (+ 1969us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:45.683822 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:45.683832 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:45.684353 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:45.685296 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:45.685320 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:45.686531 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:45.686535 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:45.687848 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:45.687860 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:45.688578 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:45.688583 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:45.688811 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:45.689530 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:45.689553 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:45.691391 (+ 1838us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:45.693504 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:45.693512 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:45.693532 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:45.693898 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:45.694212 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:45.694216 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:45.694218 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:45.694409 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:45.694898 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:45.694905 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:45.695242 (+ 337us) client_negotiation.cc:769] Sending connection context
0811 15:08:45.695468 (+ 226us) client_negotiation.cc:240] Negotiation successful
0811 15:08:45.695854 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":196,"threads_started":1}
I20260811 15:08:45.698312 8135 heartbeater.cc:347] Connected to a master server at 127.30.8.62:33265
I20260811 15:08:45.698626 8135 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.30.8.3@KRBTEST.COM; defaulting to no policy
I20260811 15:08:45.699402 8135 heartbeater.cc:511] Master 127.30.8.62:33265 requested a full tablet report, sending...
I20260811 15:08:45.700702 7588 ts_manager.cc:194] Registered new tserver with Master: 11a79800f2ea4f81a9217a4ae60f92bf (127.30.8.2:41599)
I20260811 15:08:45.701651 7588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:49073
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
I20260811 15:08:45.723865 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:33265
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:33579
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:45.846130 8140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:45.846465 8140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:45.846526 8140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:45.850785 8140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:45.850888 8140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:45.851001 8140 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:08:45.856915 8140 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:33579
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:33265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:45.858181 8140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:45.859331 8140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:45.870018 8148 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:45.870018 8147 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:45.870061 8140 server_base.cc:1034] running on GCE node
W20260811 15:08:45.870018 8150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:45.870740 8140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:45.871515 8140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:45.872684 8140 hybrid_clock.cc:648] HybridClock initialized: now 1786460925872665 us; error 37 us; skew 500 ppm
Aug 11 15:08:45 dist-test-slave-1vmg krb5kdc[7516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460925, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:45.875694 8140 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:08:45.876755 8140 webserver.cc:460] Webserver started at http://127.30.8.3:40209/ using document root <none> and password file <none>
I20260811 15:08:45.877421 8140 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:45.877491 8140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:45.877727 8140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:45.880097 8140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "9ea18cde5df645bea4bd19b8802a02e4"
format_stamp: "Formatted at 2026-08-11 15:08:45 on dist-test-slave-1vmg"
server_key: "1ddafe894f06936137a871cdeb32a991"
server_key_iv: "87fda59113f3a2dca8333a7d415304ab"
server_key_version: "encryptionkey@0"
I20260811 15:08:45.880712 8140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "9ea18cde5df645bea4bd19b8802a02e4"
format_stamp: "Formatted at 2026-08-11 15:08:45 on dist-test-slave-1vmg"
server_key: "1ddafe894f06936137a871cdeb32a991"
server_key_iv: "87fda59113f3a2dca8333a7d415304ab"
server_key_version: "encryptionkey@0"
I20260811 15:08:45.885548 8140 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 15:08:45.888646 8161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:45.890061 8140 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 15:08:45.890187 8140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "9ea18cde5df645bea4bd19b8802a02e4"
format_stamp: "Formatted at 2026-08-11 15:08:45 on dist-test-slave-1vmg"
server_key: "1ddafe894f06936137a871cdeb32a991"
server_key_iv: "87fda59113f3a2dca8333a7d415304ab"
server_key_version: "encryptionkey@0"
I20260811 15:08:45.890329 8140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:45.921231 8140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:45.925944 8140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:45.926210 8140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:45.927152 8140 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:45.928750 8140 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:45.928830 8140 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:45.928900 8140 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:45.928930 8140 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:45.938993 8140 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:45817
I20260811 15:08:45.939266 8348 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:45817 every 8 connection(s)
I20260811 15:08:45.940326 8140 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:08:45.944733 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 8140
I20260811 15:08:45.944876 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:08:45.945171 30752 external_mini_cluster.cc:1428] Setting key 37f0d4a3652cb94b1d825be7c11883bb
Aug 11 15:08:45 dist-test-slave-1vmg krb5kdc[7516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460925, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:08:45.955092 7663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:45.943035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:47407 (local address 127.30.8.62:33265)
0811 15:08:45.943221 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:45.943226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:45.944620 (+ 1394us) server_negotiation.cc:407] Connection header received
0811 15:08:45.945665 (+ 1045us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:45.945668 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:45.945734 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:45.945811 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:45.947150 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:45.947699 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:45.948848 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:45.949074 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:45.949164 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:45.951742 (+ 2578us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:45.951745 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:45.951747 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:45.951773 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:45.952925 (+ 1152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:45.953452 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:45.953455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:45.953456 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:45.953534 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:45.953891 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:45.953893 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:45.953894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:45.954092 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:45.954178 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:45.954745 (+ 567us) server_negotiation.cc:299] Negotiation successful
0811 15:08:45.954886 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 15:08:45.956022 8353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:45.943498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:33265 (local address 127.30.8.3:47407)
0811 15:08:45.944444 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:45.944492 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:45.945443 (+ 951us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:45.946037 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:45.946044 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:45.946430 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:45.947014 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:45.947032 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:45.947829 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:45.947832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:45.948725 (+ 893us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:45.948735 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:45.949228 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:45.949232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:45.949418 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:45.950012 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:45.950030 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:45.951566 (+ 1536us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:45.953044 (+ 1478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:45.953050 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:45.953069 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:45.953337 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:45.953626 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:45.953629 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:45.953631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:45.953785 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:45.954201 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:45.954210 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:45.954486 (+ 276us) client_negotiation.cc:769] Sending connection context
0811 15:08:45.954661 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 15:08:45.955105 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"thread_start_us":576,"threads_started":1}
I20260811 15:08:45.957214 8349 heartbeater.cc:347] Connected to a master server at 127.30.8.62:33265
I20260811 15:08:45.957446 8349 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:45.958021 8349 heartbeater.cc:511] Master 127.30.8.62:33265 requested a full tablet report, sending...
I20260811 15:08:45.959214 7588 ts_manager.cc:194] Registered new tserver with Master: 9ea18cde5df645bea4bd19b8802a02e4 (127.30.8.3:45817)
I20260811 15:08:45.959793 7588 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:47407
I20260811 15:08:45.971238 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/test-admin.keytab.
Aug 11 15:08:45 dist-test-slave-1vmg krb5kdc[7516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:45.989951 30752 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 15:08:45.993024 8361 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:08:45.991480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37468 (local address 127.30.8.1:46755)
0811 15:08:45.991825 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:45.991832 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:45.991859 (+ 27us) server_negotiation.cc:407] Connection header received
0811 15:08:45.991913 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:45.991919 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:45.992120 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:45.992263 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:45.992863 (+ 600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37468: BlockingRecv error: recv got EOF from 127.0.0.1:37468 (error 108)
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":148,"threads_started":1}
W20260811 15:08:45.993142 30752 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/krb5cc' of type FILE
Aug 11 15:08:45 dist-test-slave-1vmg krb5kdc[7516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:45.994236 30752 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 15:08:45 dist-test-slave-1vmg krb5kdc[7516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:45.995224 30752 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 15:08:45 dist-test-slave-1vmg krb5kdc[7516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.1@KRBTEST.COM
I20260811 15:08:46.007862 8361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:45.996177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37474 (local address 127.30.8.1:46755)
0811 15:08:45.996330 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:45.996333 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:45.996348 (+ 15us) server_negotiation.cc:407] Connection header received
0811 15:08:45.996463 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:45.996465 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:45.996509 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:45.996577 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:45.997256 (+ 679us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:45.998228 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:45.999522 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:46.000071 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:46.000188 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:46.003631 (+ 3443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:46.003650 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:46.003675 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:46.003722 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:46.005822 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:46.006451 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:46.006457 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:46.006458 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:46.006555 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:46.006786 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:46.006788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:46.006790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:46.007172 (+ 382us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:46.007389 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:46.007557 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 15:08:46.007711 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 15:08:46.011955 8361 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:08:46.011029 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37486 (local address 127.30.8.1:46755)
0811 15:08:46.011166 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:46.011169 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:46.011253 (+ 84us) server_negotiation.cc:407] Connection header received
0811 15:08:46.011358 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:46.011360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:46.011400 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:46.011463 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:46.011844 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37486: BlockingRecv error: recv got EOF from 127.0.0.1:37486 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 15:08:46.012153 30752 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/krb5cc' of type FILE
Aug 11 15:08:46 dist-test-slave-1vmg krb5kdc[7516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:46.013320 30752 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 15:08:46 dist-test-slave-1vmg krb5kdc[7516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:46.014240 30752 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 15:08:46 dist-test-slave-1vmg krb5kdc[7516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.1@KRBTEST.COM
I20260811 15:08:46.024407 8361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:46.015118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37498 (local address 127.30.8.1:46755)
0811 15:08:46.015243 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:46.015247 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:46.015344 (+ 97us) server_negotiation.cc:407] Connection header received
0811 15:08:46.015452 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:46.015454 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:46.015492 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:46.015554 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:46.016028 (+ 474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:46.016534 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:46.017305 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:46.017532 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:46.017606 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:46.020095 (+ 2489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:46.020099 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:46.020101 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:46.020131 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:46.022002 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:46.022804 (+ 802us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:46.022809 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:46.022810 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:46.022993 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:46.023377 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:46.023380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:46.023382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:46.023629 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:46.023808 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:46.024016 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 15:08:46.024202 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 15:08:46.027813 8361 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:08:46.026778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37500 (local address 127.30.8.1:46755)
0811 15:08:46.026979 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:46.026983 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:46.026996 (+ 13us) server_negotiation.cc:407] Connection header received
0811 15:08:46.027109 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:46.027112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:46.027159 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:46.027246 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:46.027680 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37500: BlockingRecv error: recv got EOF from 127.0.0.1:37500 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 15:08:46.027904 30752 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786460893564915-30752-0/krb5kdc/krb5cc' of type FILE
Aug 11 15:08:46 dist-test-slave-1vmg krb5kdc[7516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:46.029208 30752 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 15:08:46 dist-test-slave-1vmg krb5kdc[7516](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:46.030062 30752 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 15:08:46 dist-test-slave-1vmg krb5kdc[7516](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.1@KRBTEST.COM
I20260811 15:08:46.039093 8361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:46.030954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37504 (local address 127.30.8.1:46755)
0811 15:08:46.031069 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:46.031072 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:46.031140 (+ 68us) server_negotiation.cc:407] Connection header received
0811 15:08:46.031246 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:46.031248 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:46.031295 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:46.031383 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:46.031911 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:46.032421 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:46.033184 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:46.033405 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:46.033485 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:46.035790 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:46.035794 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:46.035796 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:46.035822 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:46.037294 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:46.037683 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:46.037686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:46.037687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:46.037770 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:46.037978 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:46.037980 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:46.037981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:46.038163 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:46.038284 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:46.038463 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 15:08:46.038803 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 15:08:46.040521 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 7696
I20260811 15:08:46.056303 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 7916
I20260811 15:08:46.076568 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 8140
I20260811 15:08:46.094357 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 7537
2026-08-11T15:08:46Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2537 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:08:46 dist-test-slave-1vmg krb5kdc[8381](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:08:46 dist-test-slave-1vmg krb5kdc[8381](info): set up 2 sockets
Aug 11 15:08:46 dist-test-slave-1vmg krb5kdc[8381](info): commencing operation
krb5kdc: starting...
W20260811 15:08:47.292414 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:08:47 dist-test-slave-1vmg krb5kdc[8381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:08:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:08:47Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.62.keytab.
Entry for principal oryx/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.62.keytab.
I20260811 15:08:47.440388 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:36293
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.62.keytab
--principal=oryx/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:38255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:36293
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:47.581864 8409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:47.582273 8409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:47.582338 8409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:47.586673 8409 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:08:47.586771 8409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:47.586800 8409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:47.586824 8409 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:08:47.586843 8409 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:08:47.593149 8409 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:38255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:36293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.62.keytab
--principal=oryx/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:36293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:47.594738 8409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:47.596307 8409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:47.607031 8421 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:47.607031 8416 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:47.607340 8409 server_base.cc:1034] running on GCE node
W20260811 15:08:47.607031 8418 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:47.608268 8409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:47.609694 8409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:47.610857 8409 hybrid_clock.cc:648] HybridClock initialized: now 1786460927610835 us; error 42 us; skew 500 ppm
Aug 11 15:08:47 dist-test-slave-1vmg krb5kdc[8381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460927, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:47.613945 8409 init.cc:377] Logged in from keytab as oryx/127.30.8.62@KRBTEST.COM (short username oryx)
I20260811 15:08:47.615099 8409 webserver.cc:460] Webserver started at http://127.30.8.62:42003/ using document root <none> and password file <none>
I20260811 15:08:47.615736 8409 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:47.615800 8409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:47.615999 8409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:47.618327 8409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "c7378c939fdc4fd38b8eaab16d60d54f"
format_stamp: "Formatted at 2026-08-11 15:08:47 on dist-test-slave-1vmg"
server_key: "59111e2e0367ab3a8222df3e210f199d"
server_key_iv: "02732d9ca74d850fae8e94e67ffe51f8"
server_key_version: "encryptionkey@0"
I20260811 15:08:47.618958 8409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "c7378c939fdc4fd38b8eaab16d60d54f"
format_stamp: "Formatted at 2026-08-11 15:08:47 on dist-test-slave-1vmg"
server_key: "59111e2e0367ab3a8222df3e210f199d"
server_key_iv: "02732d9ca74d850fae8e94e67ffe51f8"
server_key_version: "encryptionkey@0"
I20260811 15:08:47.623384 8409 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 15:08:47.626151 8435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:47.627620 8409 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:08:47.627754 8409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "c7378c939fdc4fd38b8eaab16d60d54f"
format_stamp: "Formatted at 2026-08-11 15:08:47 on dist-test-slave-1vmg"
server_key: "59111e2e0367ab3a8222df3e210f199d"
server_key_iv: "02732d9ca74d850fae8e94e67ffe51f8"
server_key_version: "encryptionkey@0"
I20260811 15:08:47.627866 8409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:47.642493 8409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:47.645732 8409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:47.645920 8409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:47.653501 8409 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:36293
I20260811 15:08:47.653883 8516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:36293 every 8 connection(s)
I20260811 15:08:47.654657 8409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:08:47.657976 8517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:47.660297 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 8409
I20260811 15:08:47.660422 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:08:47.660683 30752 external_mini_cluster.cc:1428] Setting key 733b3404294d8110a808f5140b2533b7
I20260811 15:08:47.664778 8517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f: Bootstrap starting.
I20260811 15:08:47.667320 8517 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:47.668143 8517 log.cc:824] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f: Log is configured to *not* fsync() on all Append() calls
Aug 11 15:08:47 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM
I20260811 15:08:47.670547 8517 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f: No bootstrap required, opened a new log
I20260811 15:08:47.673964 8517 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } }
I20260811 15:08:47.674249 8517 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:47.674306 8517 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7378c939fdc4fd38b8eaab16d60d54f, State: Initialized, Role: FOLLOWER
I20260811 15:08:47.674964 8517 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } }
I20260811 15:08:47.675158 8517 raft_consensus.cc:388] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:08:47.675226 8517 raft_consensus.cc:482] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:08:47.675335 8517 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:47.676236 8520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:47.662295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41386 (local address 127.30.8.62:36293)
0811 15:08:47.662797 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:47.662814 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:47.662854 (+ 40us) server_negotiation.cc:407] Connection header received
0811 15:08:47.663769 (+ 915us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:47.663792 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:47.664170 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:47.664522 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:47.665367 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:47.666395 (+ 1028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:47.667325 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:47.667892 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:47.668056 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:47.670778 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:47.670784 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:47.670801 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:47.670832 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:47.673016 (+ 2184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:47.673493 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:47.673498 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:47.673504 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:47.673633 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:47.673925 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:47.673928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:47.673929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:47.674449 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:47.674669 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:47.675003 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 15:08:47.675364 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":140,"threads_started":1}
I20260811 15:08:47.676491 8517 raft_consensus.cc:504] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } }
I20260811 15:08:47.676882 8517 leader_election.cc:302] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c7378c939fdc4fd38b8eaab16d60d54f; no voters:
I20260811 15:08:47.677259 8517 leader_election.cc:288] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:08:47.677357 8525 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:47.677528 8525 raft_consensus.cc:686] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [term 1 LEADER]: Becoming Leader. State: Replica: c7378c939fdc4fd38b8eaab16d60d54f, State: Running, Role: LEADER
I20260811 15:08:47.677855 8525 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } }
I20260811 15:08:47.678665 8517 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:08:47.679791 8528 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } }
I20260811 15:08:47.679929 8528 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:47.680310 8527 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7378c939fdc4fd38b8eaab16d60d54f. Latest consensus state: current_term: 1 leader_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } }
I20260811 15:08:47.680451 8527 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:47.680364 8540 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:08:47.683748 8540 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:08:47.690670 8540 catalog_manager.cc:1347] Generated new cluster ID: 035c4a9125c04bba9b2b98a3ebcd918c
I20260811 15:08:47.690750 8540 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:08:47.702389 8540 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:08:47.703595 8540 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:08:47.708671 8540 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f: Generated new TSK 0
I20260811 15:08:47.709584 8540 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.1.keytab.
Entry for principal oryx/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.1.keytab.
I20260811 15:08:47.749719 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:36293
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.1.keytab
--principal=oryx/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:38255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:47.880044 8558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:47.880447 8558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:47.880501 8558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:47.884744 8558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:47.884824 8558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:47.884889 8558 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:08:47.890846 8558 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:38255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.1.keytab
--principal=oryx/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:36293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:47.892140 8558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:47.893256 8558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:47.904238 8568 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:47.904280 8576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:47.904281 8570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:47.904542 8558 server_base.cc:1034] running on GCE node
I20260811 15:08:47.905138 8558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:47.905900 8558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:47.907060 8558 hybrid_clock.cc:648] HybridClock initialized: now 1786460927907035 us; error 39 us; skew 500 ppm
Aug 11 15:08:47 dist-test-slave-1vmg krb5kdc[8381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460927, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:47.910048 8558 init.cc:377] Logged in from keytab as oryx/127.30.8.1@KRBTEST.COM (short username oryx)
I20260811 15:08:47.911134 8558 webserver.cc:460] Webserver started at http://127.30.8.1:37955/ using document root <none> and password file <none>
I20260811 15:08:47.911755 8558 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:47.911831 8558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:47.912051 8558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:47.914299 8558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "8b439d6b462a49188ab751501b266fdd"
format_stamp: "Formatted at 2026-08-11 15:08:47 on dist-test-slave-1vmg"
server_key: "c49d8c5eab4caac923213147c28953c9"
server_key_iv: "55e9a590e619c76204df8713862adba0"
server_key_version: "encryptionkey@0"
I20260811 15:08:47.914908 8558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "8b439d6b462a49188ab751501b266fdd"
format_stamp: "Formatted at 2026-08-11 15:08:47 on dist-test-slave-1vmg"
server_key: "c49d8c5eab4caac923213147c28953c9"
server_key_iv: "55e9a590e619c76204df8713862adba0"
server_key_version: "encryptionkey@0"
I20260811 15:08:47.919634 8558 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 15:08:47.923019 8593 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:47.924700 8558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:08:47.924880 8558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "8b439d6b462a49188ab751501b266fdd"
format_stamp: "Formatted at 2026-08-11 15:08:47 on dist-test-slave-1vmg"
server_key: "c49d8c5eab4caac923213147c28953c9"
server_key_iv: "55e9a590e619c76204df8713862adba0"
server_key_version: "encryptionkey@0"
I20260811 15:08:47.925038 8558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:47.955299 8558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:47.958577 8558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:47.958794 8558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:47.959506 8558 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:47.960497 8558 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:47.960547 8558 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:47.960592 8558 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:47.960621 8558 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:47.969910 8558 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:37731
I20260811 15:08:47.969964 8775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:37731 every 8 connection(s)
I20260811 15:08:47.970980 8558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:08:47.978883 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 8558
I20260811 15:08:47.979034 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:08:47.979357 30752 external_mini_cluster.cc:1428] Setting key eeb7a674816680e3090b1b6de8a379e3
Aug 11 15:08:47 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460927, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM
I20260811 15:08:47.986524 8520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:47.973119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:50189 (local address 127.30.8.62:36293)
0811 15:08:47.973292 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:47.973296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:47.974252 (+ 956us) server_negotiation.cc:407] Connection header received
0811 15:08:47.975755 (+ 1503us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:47.975758 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:47.975813 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:47.975896 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:47.977208 (+ 1312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:47.977772 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:47.979143 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:47.979479 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:47.979589 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:47.982698 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:47.982703 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:47.982706 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:47.982736 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:47.984155 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:47.984824 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:47.984826 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:47.984827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:47.984903 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:47.985283 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:47.985285 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:47.985286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:47.985480 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:47.985555 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:47.986260 (+ 705us) server_negotiation.cc:299] Negotiation successful
0811 15:08:47.986379 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 15:08:47.987989 8781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:47.973521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.1:50189)
0811 15:08:47.974072 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:47.974119 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:47.975518 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:47.976077 (+ 559us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:47.976085 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:47.976508 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:47.977052 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:47.977069 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:47.977883 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:47.977886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:47.978981 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:47.978995 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:47.979672 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:47.979677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:47.979930 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:47.980599 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:47.980621 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:47.982484 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:47.984296 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:47.984303 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:47.984325 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:47.984711 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:47.984989 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:47.984992 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:47.984994 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:47.985197 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:47.985586 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:47.985594 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:47.985979 (+ 385us) client_negotiation.cc:769] Sending connection context
0811 15:08:47.986250 (+ 271us) client_negotiation.cc:240] Negotiation successful
0811 15:08:47.986678 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"thread_start_us":136,"threads_started":1}
I20260811 15:08:47.989513 8776 heartbeater.cc:347] Connected to a master server at 127.30.8.62:36293
I20260811 15:08:47.989789 8776 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:47.990351 8776 heartbeater.cc:511] Master 127.30.8.62:36293 requested a full tablet report, sending...
WARNING: no policy specified for oryx/127.30.8.2@KRBTEST.COM; defaulting to no policy
I20260811 15:08:47.992260 8461 ts_manager.cc:194] Registered new tserver with Master: 8b439d6b462a49188ab751501b266fdd (127.30.8.1:37731)
I20260811 15:08:47.993592 8461 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:50189
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.8.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.2.keytab.
Entry for principal oryx/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.2.keytab.
I20260811 15:08:48.011258 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:36293
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.2.keytab
--principal=oryx/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:38255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:48.135579 8784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:48.135895 8784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:48.135946 8784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:48.140287 8784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:48.140383 8784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:48.140462 8784 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:08:48.146286 8784 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:38255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.2.keytab
--principal=oryx/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:36293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:48.147645 8784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:48.148739 8784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:48.159165 8793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:48.159165 8794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:48.159260 8796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:48.159253 8784 server_base.cc:1034] running on GCE node
I20260811 15:08:48.159917 8784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:48.160661 8784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:48.161852 8784 hybrid_clock.cc:648] HybridClock initialized: now 1786460928161803 us; error 61 us; skew 500 ppm
Aug 11 15:08:48 dist-test-slave-1vmg krb5kdc[8381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460928, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:48.164915 8784 init.cc:377] Logged in from keytab as oryx/127.30.8.2@KRBTEST.COM (short username oryx)
I20260811 15:08:48.165953 8784 webserver.cc:460] Webserver started at http://127.30.8.2:44135/ using document root <none> and password file <none>
I20260811 15:08:48.166568 8784 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:48.166653 8784 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:48.166877 8784 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:48.169203 8784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "fea2d50e2519476984b8fec3e8d172bc"
format_stamp: "Formatted at 2026-08-11 15:08:48 on dist-test-slave-1vmg"
server_key: "f28adf8c2e65e5b4aeceab9e4a837e11"
server_key_iv: "fd04548ba8fc1b1d22f83cb475246db9"
server_key_version: "encryptionkey@0"
I20260811 15:08:48.169812 8784 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "fea2d50e2519476984b8fec3e8d172bc"
format_stamp: "Formatted at 2026-08-11 15:08:48 on dist-test-slave-1vmg"
server_key: "f28adf8c2e65e5b4aeceab9e4a837e11"
server_key_iv: "fd04548ba8fc1b1d22f83cb475246db9"
server_key_version: "encryptionkey@0"
I20260811 15:08:48.174016 8784 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 15:08:48.176815 8807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:48.178161 8784 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:08:48.178296 8784 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "fea2d50e2519476984b8fec3e8d172bc"
format_stamp: "Formatted at 2026-08-11 15:08:48 on dist-test-slave-1vmg"
server_key: "f28adf8c2e65e5b4aeceab9e4a837e11"
server_key_iv: "fd04548ba8fc1b1d22f83cb475246db9"
server_key_version: "encryptionkey@0"
I20260811 15:08:48.178416 8784 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:48.196831 8784 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:48.200623 8784 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:48.200845 8784 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:48.201505 8784 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:48.202454 8784 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:48.202508 8784 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:48.202566 8784 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:48.202605 8784 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:48.214546 8784 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:41249
I20260811 15:08:48.214705 8999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:41249 every 8 connection(s)
I20260811 15:08:48.215950 8784 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:08:48.220168 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 8784
I20260811 15:08:48.220323 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:08:48.220693 30752 external_mini_cluster.cc:1428] Setting key d8a0f5a6044fcf9e84e481b460a9543b
Aug 11 15:08:48 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460928, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM
WARNING: no policy specified for oryx/127.30.8.3@KRBTEST.COM; defaulting to no policy
I20260811 15:08:48.235954 8520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:48.218470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:37803 (local address 127.30.8.62:36293)
0811 15:08:48.218725 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:48.218731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:48.219808 (+ 1077us) server_negotiation.cc:407] Connection header received
0811 15:08:48.221885 (+ 2077us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:48.221889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:48.221965 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:48.222065 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:48.224068 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.224637 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.226427 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.226691 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.226812 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:48.230658 (+ 3846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:48.230664 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:48.230667 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:48.230709 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:48.233124 (+ 2415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:48.233951 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:48.233955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:48.233958 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:48.234066 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:48.234498 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:48.234501 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:48.234503 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:48.234814 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:48.234968 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:48.235683 (+ 715us) server_negotiation.cc:299] Negotiation successful
0811 15:08:48.235817 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106}
I20260811 15:08:48.236930 9004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:48.218829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.2:37803)
0811 15:08:48.219555 (+ 726us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:48.219619 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:48.221572 (+ 1953us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:48.222335 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:48.222347 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:48.222954 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:48.223861 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:48.223886 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.224803 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.224806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:48.226245 (+ 1439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:48.226258 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.226892 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.226896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:48.227151 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:48.228143 (+ 992us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:48.228171 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:48.230293 (+ 2122us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:48.233317 (+ 3024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:48.233325 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:48.233415 (+ 90us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:48.233790 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:48.234189 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:48.234191 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:48.234193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:48.234367 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:48.235034 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:48.235042 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:48.235378 (+ 336us) client_negotiation.cc:769] Sending connection context
0811 15:08:48.235591 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 15:08:48.235930 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":158,"threads_started":1}
I20260811 15:08:48.238350 9001 heartbeater.cc:347] Connected to a master server at 127.30.8.62:36293
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.8.3@KRBTEST.COM" created.
I20260811 15:08:48.238624 9001 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:48.239256 9001 heartbeater.cc:511] Master 127.30.8.62:36293 requested a full tablet report, sending...
I20260811 15:08:48.240513 8461 ts_manager.cc:194] Registered new tserver with Master: fea2d50e2519476984b8fec3e8d172bc (127.30.8.2:41249)
I20260811 15:08:48.241154 8461 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:37803
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.3.keytab.
Entry for principal oryx/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.3.keytab.
I20260811 15:08:48.258361 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:36293
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.3.keytab
--principal=oryx/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:38255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:48.395346 9007 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:48.395684 9007 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:48.395730 9007 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:48.400071 9007 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:48.400161 9007 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:48.400228 9007 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:08:48.405988 9007 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:38255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.3.keytab
--principal=oryx/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:36293
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9007
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:48.407351 9007 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:48.408527 9007 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:48.419185 9015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:48.419144 9018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:48.419144 9013 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:48.419409 9007 server_base.cc:1034] running on GCE node
I20260811 15:08:48.419955 9007 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:48.420693 9007 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:48.421864 9007 hybrid_clock.cc:648] HybridClock initialized: now 1786460928421849 us; error 40 us; skew 500 ppm
Aug 11 15:08:48 dist-test-slave-1vmg krb5kdc[8381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460928, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:48.424821 9007 init.cc:377] Logged in from keytab as oryx/127.30.8.3@KRBTEST.COM (short username oryx)
I20260811 15:08:48.425915 9007 webserver.cc:460] Webserver started at http://127.30.8.3:34479/ using document root <none> and password file <none>
I20260811 15:08:48.426556 9007 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:48.426632 9007 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:48.426855 9007 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:48.429317 9007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "0f34c101cebf4318a7484b40cdfc072a"
format_stamp: "Formatted at 2026-08-11 15:08:48 on dist-test-slave-1vmg"
server_key: "4a4237d6e2f04f770ac66096ed1a3d40"
server_key_iv: "39fb3a37a23e2e762dc8e99d5a5645ff"
server_key_version: "encryptionkey@0"
I20260811 15:08:48.429967 9007 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "0f34c101cebf4318a7484b40cdfc072a"
format_stamp: "Formatted at 2026-08-11 15:08:48 on dist-test-slave-1vmg"
server_key: "4a4237d6e2f04f770ac66096ed1a3d40"
server_key_iv: "39fb3a37a23e2e762dc8e99d5a5645ff"
server_key_version: "encryptionkey@0"
I20260811 15:08:48.434482 9007 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 15:08:48.437748 9029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:48.439355 9007 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 15:08:48.439502 9007 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "0f34c101cebf4318a7484b40cdfc072a"
format_stamp: "Formatted at 2026-08-11 15:08:48 on dist-test-slave-1vmg"
server_key: "4a4237d6e2f04f770ac66096ed1a3d40"
server_key_iv: "39fb3a37a23e2e762dc8e99d5a5645ff"
server_key_version: "encryptionkey@0"
I20260811 15:08:48.439635 9007 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:48.474704 9007 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:48.477938 9007 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:48.478168 9007 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:48.478874 9007 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:48.479889 9007 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:48.479944 9007 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:48.480003 9007 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:48.480032 9007 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:48.489606 9007 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:45381
I20260811 15:08:48.489634 9225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:45381 every 8 connection(s)
I20260811 15:08:48.490676 9007 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:08:48.499275 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 9007
I20260811 15:08:48.499406 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:08:48.499699 30752 external_mini_cluster.cc:1428] Setting key 60681dfcc8da655d20ec4abcc730176a
Aug 11 15:08:48 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460928, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM
I20260811 15:08:48.505405 8520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:48.492814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:36553 (local address 127.30.8.62:36293)
0811 15:08:48.492977 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:48.492983 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:48.493913 (+ 930us) server_negotiation.cc:407] Connection header received
0811 15:08:48.495403 (+ 1490us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:48.495406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:48.495458 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:48.495538 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:48.496860 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.497400 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.498557 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.498775 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.498847 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:48.501869 (+ 3022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:48.501874 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:48.501877 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:48.501907 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:48.503160 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:48.503704 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:48.503707 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:48.503708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:48.503790 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:48.504156 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:48.504160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:48.504161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:48.504374 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:48.504469 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:48.505090 (+ 621us) server_negotiation.cc:299] Negotiation successful
0811 15:08:48.505235 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 15:08:48.506347 9230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:48.493230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.3:36553)
0811 15:08:48.493739 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:48.493787 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:48.495179 (+ 1392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:48.495732 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:48.495740 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:48.496172 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:48.496730 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:48.496745 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.497508 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.497511 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:48.498451 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:48.498462 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.498939 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.498944 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:48.499131 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:48.499819 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:48.499844 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:48.501686 (+ 1842us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:48.503282 (+ 1596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:48.503290 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:48.503312 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:48.503592 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:48.503890 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:48.503893 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:48.503896 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:48.504053 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:48.504489 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:48.504497 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:48.504820 (+ 323us) client_negotiation.cc:769] Sending connection context
0811 15:08:48.505018 (+ 198us) client_negotiation.cc:240] Negotiation successful
0811 15:08:48.505418 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":116,"threads_started":1}
I20260811 15:08:48.507581 9226 heartbeater.cc:347] Connected to a master server at 127.30.8.62:36293
I20260811 15:08:48.507856 9226 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:48.508427 9226 heartbeater.cc:511] Master 127.30.8.62:36293 requested a full tablet report, sending...
I20260811 15:08:48.509585 8461 ts_manager.cc:194] Registered new tserver with Master: 0f34c101cebf4318a7484b40cdfc072a (127.30.8.3:45381)
I20260811 15:08:48.510179 8461 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:36553
I20260811 15:08:48.513980 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.8.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.8.61@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.8.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.61.keytab.
Entry for principal oryx/127.30.8.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.61.keytab.
I20260811 15:08:48.546150 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.61:40131
--webserver_interface=127.30.8.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.61.keytab
--principal=oryx/127.30.8.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:38255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:36293,127.30.8.61:40131
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:48.675218 9234 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:48.675539 9234 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:48.675582 9234 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:48.680004 9234 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:08:48.680089 9234 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:48.680107 9234 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:48.680131 9234 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:08:48.680146 9234 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:08:48.686148 9234 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:38255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:36293,127.30.8.61:40131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.61.keytab
--principal=oryx/127.30.8.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.61:40131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.8.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9234
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
W20260811 15:08:48.686408 9234 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.30.8.62:36293,127.30.8.61:40131'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 15:08:48.687439 9234 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:48.688479 9234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:48.697634 9243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:48.697674 9240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:48.697798 9234 server_base.cc:1034] running on GCE node
W20260811 15:08:48.697866 9241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:48.698593 9234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:48.699925 9234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:48.701128 9234 hybrid_clock.cc:648] HybridClock initialized: now 1786460928701097 us; error 31 us; skew 500 ppm
Aug 11 15:08:48 dist-test-slave-1vmg krb5kdc[8381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460928, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:48.704627 9234 init.cc:377] Logged in from keytab as oryx/127.30.8.61@KRBTEST.COM (short username oryx)
I20260811 15:08:48.706089 9234 webserver.cc:460] Webserver started at http://127.30.8.61:44977/ using document root <none> and password file <none>
I20260811 15:08:48.706853 9234 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:48.706977 9234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:48.707273 9234 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:48.710031 9234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/data/instance:
uuid: "dd9736da73324408b58ba279deb3f85e"
format_stamp: "Formatted at 2026-08-11 15:08:48 on dist-test-slave-1vmg"
server_key: "6d7671cf9f2df2cd57611b6e09786b42"
server_key_iv: "ce4ea24e21203a004a8a57aa88416164"
server_key_version: "encryptionkey@0"
I20260811 15:08:48.711217 9234 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/wal/instance:
uuid: "dd9736da73324408b58ba279deb3f85e"
format_stamp: "Formatted at 2026-08-11 15:08:48 on dist-test-slave-1vmg"
server_key: "6d7671cf9f2df2cd57611b6e09786b42"
server_key_iv: "ce4ea24e21203a004a8a57aa88416164"
server_key_version: "encryptionkey@0"
I20260811 15:08:48.715935 9234 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 15:08:48.718959 9252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:48.720371 9234 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 15:08:48.720554 9234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/wal
uuid: "dd9736da73324408b58ba279deb3f85e"
format_stamp: "Formatted at 2026-08-11 15:08:48 on dist-test-slave-1vmg"
server_key: "6d7671cf9f2df2cd57611b6e09786b42"
server_key_iv: "ce4ea24e21203a004a8a57aa88416164"
server_key_version: "encryptionkey@0"
I20260811 15:08:48.720713 9234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:48.746065 9234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:48.749883 9234 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:48.750080 9234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:48.757668 9234 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.61:40131
I20260811 15:08:48.757931 9341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.61:40131 every 8 connection(s)
I20260811 15:08:48.758792 9234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/data/info.pb
I20260811 15:08:48.761989 9343 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:48.765511 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 9234
I20260811 15:08:48.766503 9343 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 15:08:48 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460928, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.61@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM
Aug 11 15:08:48 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.8.61@KRBTEST.COM
I20260811 15:08:48.780222 8520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:48.768669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41392 (local address 127.30.8.62:36293)
0811 15:08:48.768802 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:48.768807 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:48.769126 (+ 319us) server_negotiation.cc:407] Connection header received
0811 15:08:48.769448 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:48.769451 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:48.769503 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:48.769596 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:48.770680 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.771242 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.772360 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.772573 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.772653 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:48.775748 (+ 3095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:48.775751 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:48.775754 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:48.775781 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:48.777454 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:48.778128 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:48.778130 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:48.778132 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:48.778231 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:48.778724 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:48.778728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:48.778730 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:48.779092 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:48.779206 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:48.779856 (+ 650us) server_negotiation.cc:299] Negotiation successful
0811 15:08:48.780022 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 15:08:48.781194 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:48.767487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52342 (local address 127.30.8.61:40131)
0811 15:08:48.767999 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:48.768014 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:48.768051 (+ 37us) server_negotiation.cc:407] Connection header received
0811 15:08:48.768841 (+ 790us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:48.768868 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:48.769263 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:48.769509 (+ 246us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:48.770260 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.771490 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.772343 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.772767 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.772951 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:48.776217 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:48.776224 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:48.776238 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:48.776261 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:48.778032 (+ 1771us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:48.778593 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:48.778601 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:48.778602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:48.778687 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:48.778979 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:48.778983 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:48.778985 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:48.779566 (+ 581us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:48.779782 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:48.780036 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 15:08:48.780399 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":215,"server-negotiator.queue_time_us":358,"spinlock_wait_cycles":40448,"thread_start_us":151,"threads_started":1}
I20260811 15:08:48.781203 9351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:48.768502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.0.0.1:41392)
0811 15:08:48.768963 (+ 461us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:48.769016 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:48.769246 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:48.769673 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:48.769678 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:48.770257 (+ 579us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:48.770520 (+ 263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:48.770534 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.771372 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.771375 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:48.772228 (+ 853us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:48.772238 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.772728 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.772732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:48.772968 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:48.773823 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:48.773842 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:48.775585 (+ 1743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:48.777587 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:48.777597 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:48.777609 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:48.778018 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:48.778364 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:48.778368 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:48.778370 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:48.778610 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:48.779223 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:48.779231 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:48.779561 (+ 330us) client_negotiation.cc:769] Sending connection context
0811 15:08:48.779801 (+ 240us) client_negotiation.cc:240] Negotiation successful
0811 15:08:48.780195 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":258,"mutex_wait_us":65,"spinlock_wait_cycles":26112,"thread_start_us":156,"threads_started":1}
I20260811 15:08:48.783806 9343 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 15:08:48 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460928, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.61@KRBTEST.COM for oryx/127.30.8.61@KRBTEST.COM
I20260811 15:08:48.792357 9351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:48.784363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:40131 (local address 127.0.0.1:52356)
0811 15:08:48.784559 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:48.784575 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:48.784656 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:48.784865 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:48.784867 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:48.785047 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:48.785164 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:48.785169 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.785864 (+ 695us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.785867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:48.786418 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:48.786428 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.787566 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.787569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:48.787678 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:48.788229 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:48.788242 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:48.789521 (+ 1279us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:48.791038 (+ 1517us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:48.791041 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:48.791042 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:48.791249 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:48.791480 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:48.791482 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:48.791483 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:48.791558 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:48.791881 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:48.791883 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:48.791972 (+ 89us) client_negotiation.cc:769] Sending connection context
0811 15:08:48.792069 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 15:08:48.792208 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":114,"mutex_wait_us":15}
I20260811 15:08:48.792377 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:48.784500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52356 (local address 127.30.8.61:40131)
0811 15:08:48.784621 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:48.784624 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:48.784636 (+ 12us) server_negotiation.cc:407] Connection header received
0811 15:08:48.784736 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:48.784738 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:48.784773 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:48.784840 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:48.785255 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.785769 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.786526 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.787458 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.787554 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:48.789659 (+ 2105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:48.789662 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:48.789664 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:48.789686 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:48.790887 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:48.791332 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:48.791334 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:48.791336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:48.791406 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:48.791627 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:48.791629 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:48.791630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:48.791809 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:48.791882 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:48.792110 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 15:08:48.792260 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":53}
I20260811 15:08:48.795629 9343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: Bootstrap starting.
I20260811 15:08:48.798207 9343 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:48.799072 9343 log.cc:824] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:48.801288 9343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: No bootstrap required, opened a new log
I20260811 15:08:48.804514 9343 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } }
I20260811 15:08:48.804780 9343 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:48.804842 9343 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd9736da73324408b58ba279deb3f85e, State: Initialized, Role: FOLLOWER
I20260811 15:08:48.805341 9343 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } }
I20260811 15:08:48.805910 9353 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } } }
I20260811 15:08:48.806068 9353 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:08:48.806689 9343 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: configured and running, proceeding with master startup.
W20260811 15:08:48.811717 9371 catalog_manager.cc:1558] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:08:48.811825 9371 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:08:48.814729 9234 master_runner.cc:428] Detected that this master dd9736da73324408b58ba279deb3f85e is joining an existing cluster
I20260811 15:08:48.814810 9234 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.8.61:40131
I20260811 15:08:48.826258 8520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:48.817322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41408 (local address 127.30.8.62:36293)
0811 15:08:48.817463 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:48.817469 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:48.817727 (+ 258us) server_negotiation.cc:407] Connection header received
0811 15:08:48.817950 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:48.817953 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:48.818017 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:48.818073 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:48.818711 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.819299 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.820552 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.820900 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.821029 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:48.822389 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:48.822394 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:48.822397 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:48.822427 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:48.824098 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:48.824669 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:48.824673 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:48.824675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:48.824945 (+ 270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:48.825301 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:48.825305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:48.825307 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:48.825558 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:48.825654 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:48.825943 (+ 289us) server_negotiation.cc:299] Negotiation successful
0811 15:08:48.826084 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 15:08:48.826303 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:48.817677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52366 (local address 127.30.8.61:40131)
0811 15:08:48.817776 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:48.817781 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:48.817795 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:08:48.817929 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:48.817933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:48.817977 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:48.818064 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:48.818710 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.819536 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.820610 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.820820 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.820918 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:48.822224 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:48.822229 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:48.822231 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:48.822252 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:48.824066 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:48.824671 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:48.824674 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:48.824675 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:48.824803 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:48.825211 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:48.825215 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:48.825217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:48.825459 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:48.825570 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:48.826038 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 15:08:48.826145 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 15:08:48.826319 9383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:48.817200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.0.0.1:41408)
0811 15:08:48.817637 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:48.817653 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:48.817826 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:48.818221 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:48.818223 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:48.818417 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:48.818574 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:48.818580 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.819420 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.819424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:48.820399 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:48.820411 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.821168 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.821172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:48.821274 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:48.821737 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:48.821752 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:48.822232 (+ 480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:48.824220 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:48.824223 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:48.824226 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:48.824536 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:48.825051 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:48.825054 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:48.825057 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:48.825161 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:48.825650 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:48.825652 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:48.825737 (+ 85us) client_negotiation.cc:769] Sending connection context
0811 15:08:48.825882 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 15:08:48.826061 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":5,"spinlock_wait_cycles":16896,"thread_start_us":135,"threads_started":1}
I20260811 15:08:48.826835 9381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:48.817243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:40131 (local address 127.0.0.1:52366)
0811 15:08:48.817681 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:48.817695 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:48.817823 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:48.818098 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:48.818101 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:48.818365 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:48.818589 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:48.818598 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.819669 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.819672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:48.820502 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:48.820511 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.820949 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.820954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:48.821046 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:48.821590 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:48.821610 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:48.822083 (+ 473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:48.824199 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:48.824203 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:48.824206 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:48.824559 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:48.824898 (+ 339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:48.824901 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:48.824902 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:48.824998 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:48.825574 (+ 576us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:48.825578 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:48.825699 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 15:08:48.826544 (+ 845us) client_negotiation.cc:240] Negotiation successful
0811 15:08:48.826698 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"mutex_wait_us":44,"spinlock_wait_cycles":18560,"thread_start_us":131,"threads_started":1}
Aug 11 15:08:48 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460927, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.62@KRBTEST.COM for oryx/127.30.8.61@KRBTEST.COM
I20260811 15:08:48.843797 9350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:48.832319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52380 (local address 127.30.8.61:40131)
0811 15:08:48.832475 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:48.832480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:48.832796 (+ 316us) server_negotiation.cc:407] Connection header received
0811 15:08:48.832981 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:48.832984 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:48.833035 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:48.833102 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:48.833995 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.834796 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.836197 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.836534 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.836610 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:48.839916 (+ 3306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:48.839920 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:48.839922 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:48.839950 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:48.841445 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:48.842111 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:48.842115 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:48.842117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:48.842221 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:48.842565 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:48.842568 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:48.842570 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:48.842821 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:48.843032 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:48.843421 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 15:08:48.843620 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 15:08:48.843811 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:48.832260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:40131 (local address 127.0.0.1:52380)
0811 15:08:48.832649 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:48.832704 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:48.832856 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:48.833162 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:48.833168 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:48.833662 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:48.833850 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:48.833860 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.834978 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.834982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:48.836052 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:48.836066 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.836786 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.836790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:48.836908 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:48.837550 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:48.837566 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:48.839680 (+ 2114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:48.841596 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:48.841611 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:48.841621 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:48.841975 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:48.842345 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:48.842348 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:48.842351 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:48.842458 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:48.842977 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:48.842987 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:48.843160 (+ 173us) client_negotiation.cc:769] Sending connection context
0811 15:08:48.843344 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 15:08:48.843620 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for oryx/127.30.8.60@KRBTEST.COM; defaulting to no policy
I20260811 15:08:48.845724 8461 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.8.61:40131: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "dd9736da73324408b58ba279deb3f85e"
member_type: NON_VOTER
last_known_addr {
host: "127.30.8.61"
port: 40131
}
attrs {
promote: true
}
}
dest_uuid: "c7378c939fdc4fd38b8eaab16d60d54f"
cas_config_opid_index: -1
I20260811 15:08:48.846449 8461 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: NON_VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: true } }
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.8.60@KRBTEST.COM" created.
I20260811 15:08:48.849341 9295 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:48.850584 9295 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 1 FOLLOWER]: Refusing update from remote peer c7378c939fdc4fd38b8eaab16d60d54f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 15:08:48.851080 9391 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER dd9736da73324408b58ba279deb3f85e (127.30.8.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: NON_VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: true } } }
I20260811 15:08:48.851774 9397 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: NON_VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:48.853215 9391 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.8.61:40131
I20260811 15:08:48.853853 9391 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: NON_VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: true } } }
I20260811 15:08:48.853968 9391 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:48.858994 9234 master.cc:537] Master@127.30.8.61:40131 shutting down...
I20260811 15:08:48.859131 9401 mvcc.cc:205] Tried to move back new op lower bound from 7317343959808913408 to 7317343959769354240. Current Snapshot: MvccSnapshot[applied={T|T < 7317343959808913408}]
I20260811 15:08:48.860913 9295 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 1 LEARNER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER dd9736da73324408b58ba279deb3f85e (127.30.8.61) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: NON_VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: true } } }
I20260811 15:08:48.863524 9353 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7378c939fdc4fd38b8eaab16d60d54f. Latest consensus state: current_term: 1 leader_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: NON_VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: true } } }
I20260811 15:08:48.863706 9353 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: This master's current role is: LEARNER
I20260811 15:08:48.863844 9353 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: NON_VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: true } } }
I20260811 15:08:48.863982 9353 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: This master's current role is: LEARNER
I20260811 15:08:48.864223 9406 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: NON_VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: true } } }
I20260811 15:08:48.864318 9406 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: This master's current role is: LEARNER
I20260811 15:08:48.866436 9234 raft_consensus.cc:2229] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 1 LEARNER]: Raft consensus shutting down.
I20260811 15:08:48.866770 9234 raft_consensus.cc:2258] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 1 LEARNER]: Raft consensus is shut down!
I20260811 15:08:48.866852 9234 tablet_replica.cc:330] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: stopping tablet replica
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.8.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.60.keytab.
Entry for principal oryx/127.30.8.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.60.keytab.
I20260811 15:08:48.874017 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.60:33179
--webserver_interface=127.30.8.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.60.keytab
--principal=oryx/127.30.8.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:38255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:36293,127.30.8.61:40131,127.30.8.60:33179
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:08:48.883757 9234 master.cc:552] Master@127.30.8.61:40131 shutdown complete.
I20260811 15:08:48.883845 9234 master_runner.cc:305] Clearing existing system tablet
I20260811 15:08:48.884135 9234 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:48.884202 9234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:48.885689 9234 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.001s
I20260811 15:08:48.886750 9416 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:48.887054 9234 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:48.887197 9234 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:48.887244 9234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/wal
uuid: "dd9736da73324408b58ba279deb3f85e"
format_stamp: "Formatted at 2026-08-11 15:08:48 on dist-test-slave-1vmg"
server_key: "6d7671cf9f2df2cd57611b6e09786b42"
server_key_iv: "ce4ea24e21203a004a8a57aa88416164"
server_key_version: "encryptionkey@0"
I20260811 15:08:48.889396 9234 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:48.892149 9234 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 15:08:48.892266 9234 log.cc:1196] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 15:08:48.892591 9234 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: Deleting consensus metadata
I20260811 15:08:48.892778 9234 master_runner.cc:315] Copying system tablet from 127.30.8.62:36293
I20260811 15:08:48.894094 9234 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: tablet copy: Beginning tablet copy session from remote peer at address 127.30.8.62:36293
I20260811 15:08:48.904171 9425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:48.894758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.0.0.1:41422)
0811 15:08:48.895153 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:48.895169 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:48.895311 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:48.895612 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:48.895615 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:48.895870 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:48.896012 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:48.896020 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.897110 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.897114 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:48.898077 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:48.898091 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.898700 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.898704 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:48.898808 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:48.899551 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:48.899567 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:48.900082 (+ 515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:48.902347 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:48.902351 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:48.902354 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:48.902630 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:48.902980 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:48.902984 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:48.902986 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:48.903094 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:48.903564 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:48.903567 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:48.903683 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 15:08:48.903797 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 15:08:48.903972 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":95,"threads_started":1}
I20260811 15:08:48.904168 8520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:48.894903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41422 (local address 127.30.8.62:36293)
0811 15:08:48.895064 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:48.895069 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:48.895267 (+ 198us) server_negotiation.cc:407] Connection header received
0811 15:08:48.895413 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:48.895417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:48.895470 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:48.895575 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:48.896142 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.896974 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.898222 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:48.898561 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:48.898676 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:48.900293 (+ 1617us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:48.900297 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:48.900300 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:48.900330 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:48.902212 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:48.902746 (+ 534us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:48.902749 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:48.902751 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:48.902850 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:48.903199 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:48.903202 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:48.903204 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:48.903457 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:48.903560 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:48.903872 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 15:08:48.904008 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 15:08:48.905267 8493 tablet_copy_service.cc:140] P c7378c939fdc4fd38b8eaab16d60d54f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer dd9736da73324408b58ba279deb3f85e ({username='oryx', principal='oryx/127.30.8.61@KRBTEST.COM'} at 127.0.0.1:41422)
I20260811 15:08:48.905470 8493 tablet_copy_service.cc:161] P c7378c939fdc4fd38b8eaab16d60d54f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer dd9736da73324408b58ba279deb3f85e at {username='oryx', principal='oryx/127.30.8.61@KRBTEST.COM'} at 127.0.0.1:41422: session id = dd9736da73324408b58ba279deb3f85e-00000000000000000000000000000000
I20260811 15:08:48.907415 8493 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 15:08:48.909235 9234 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 15:08:48.911958 9234 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: tablet copy: Starting download of 0 data blocks...
I20260811 15:08:48.912178 9234 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: tablet copy: Starting download of 1 WAL segments...
I20260811 15:08:48.914378 9234 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 15:08:48.916693 8493 tablet_copy_service.cc:342] P c7378c939fdc4fd38b8eaab16d60d54f: Request end of tablet copy session dd9736da73324408b58ba279deb3f85e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.8.61@KRBTEST.COM'} at 127.0.0.1:41422
I20260811 15:08:48.916860 8493 tablet_copy_service.cc:433] P c7378c939fdc4fd38b8eaab16d60d54f: ending tablet copy session dd9736da73324408b58ba279deb3f85e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer dd9736da73324408b58ba279deb3f85e
I20260811 15:08:48.926076 9234 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:48.928058 9426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:48.928184 9430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:48.928668 9428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:48.931094 9234 server_base.cc:1034] running on GCE node
I20260811 15:08:48.931423 9234 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:48.931857 9234 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:48.933028 9234 hybrid_clock.cc:648] HybridClock initialized: now 1786460928933016 us; error 35 us; skew 500 ppm
Aug 11 15:08:48 dist-test-slave-1vmg krb5kdc[8381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460928, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:48.934795 9234 init.cc:377] Logged in from keytab as oryx/127.30.8.61@KRBTEST.COM (short username oryx)
I20260811 15:08:48.935391 9234 webserver.cc:460] Webserver started at http://127.30.8.61:42659/ using document root <none> and password file <none>
I20260811 15:08:48.935606 9234 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:48.935652 9234 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:48.937314 9234 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 15:08:48.938457 9440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:48.938732 9234 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 15:08:48.938815 9234 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/wal
uuid: "dd9736da73324408b58ba279deb3f85e"
format_stamp: "Formatted at 2026-08-11 15:08:48 on dist-test-slave-1vmg"
server_key: "6d7671cf9f2df2cd57611b6e09786b42"
server_key_iv: "ce4ea24e21203a004a8a57aa88416164"
server_key_version: "encryptionkey@0"
I20260811 15:08:48.938901 9234 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:48.962464 9234 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:48.964814 9234 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:48.970163 9234 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.61:40131
I20260811 15:08:48.970851 9234 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-1/data/info.pb
I20260811 15:08:48.971282 9534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.61:40131 every 8 connection(s)
I20260811 15:08:48.973289 9535 sys_catalog.cc:263] Verifying existing consensus state
I20260811 15:08:48.974406 9535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: Bootstrap starting.
I20260811 15:08:48.983201 9535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 15:08:48.983577 9535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: Bootstrap complete.
I20260811 15:08:48.984218 9535 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: NON_VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: true } }
I20260811 15:08:48.984359 9535 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: dd9736da73324408b58ba279deb3f85e, State: Initialized, Role: LEARNER
I20260811 15:08:48.984467 9535 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: NON_VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: true } }
I20260811 15:08:48.984972 9543 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: NON_VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: true } } }
I20260811 15:08:48.985082 9543 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: This master's current role is: LEARNER
I20260811 15:08:48.985121 9535 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:08:48.989039 9562 catalog_manager.cc:1259] Loaded cluster ID: 035c4a9125c04bba9b2b98a3ebcd918c
I20260811 15:08:48.989169 9562 catalog_manager.cc:1552] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: loading cluster ID for follower catalog manager: success
I20260811 15:08:48.991024 9562 catalog_manager.cc:1574] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: acquiring CA information for follower catalog manager: success
I20260811 15:08:48.992578 9562 catalog_manager.cc:1602] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 15:08:48.996793 8776 heartbeater.cc:503] Master 127.30.8.62:36293 was elected leader, sending a full tablet report...
W20260811 15:08:49.025585 9414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:49.025928 9414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:49.025977 9414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:49.030341 9414 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:08:49.030437 9414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:49.030457 9414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:49.030474 9414 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:08:49.030503 9414 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:08:49.036548 9414 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:38255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:36293,127.30.8.61:40131,127.30.8.60:33179
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.60.keytab
--principal=oryx/127.30.8.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.60:33179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.8.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:49.037833 9414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:49.039022 9414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:49.049250 9573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:49.049360 9570 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:49.049311 9571 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:49.049650 9414 server_base.cc:1034] running on GCE node
I20260811 15:08:49.050519 9414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:49.051885 9414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:49.053071 9414 hybrid_clock.cc:648] HybridClock initialized: now 1786460929053058 us; error 42 us; skew 500 ppm
Aug 11 15:08:49 dist-test-slave-1vmg krb5kdc[8381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460929, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:49.056236 9414 init.cc:377] Logged in from keytab as oryx/127.30.8.60@KRBTEST.COM (short username oryx)
I20260811 15:08:49.057430 9414 webserver.cc:460] Webserver started at http://127.30.8.60:39775/ using document root <none> and password file <none>
I20260811 15:08:49.058082 9414 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:49.058159 9414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:49.058391 9414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:49.060807 9414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/data/instance:
uuid: "660b3ae52063414f8fbd6024f6b8296a"
format_stamp: "Formatted at 2026-08-11 15:08:49 on dist-test-slave-1vmg"
server_key: "fd61b9ed12f8bdd40b2d26213f5a3910"
server_key_iv: "fb83dcaab09249abb20ddf45ac3efbf1"
server_key_version: "encryptionkey@0"
I20260811 15:08:49.061420 9414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/wal/instance:
uuid: "660b3ae52063414f8fbd6024f6b8296a"
format_stamp: "Formatted at 2026-08-11 15:08:49 on dist-test-slave-1vmg"
server_key: "fd61b9ed12f8bdd40b2d26213f5a3910"
server_key_iv: "fb83dcaab09249abb20ddf45ac3efbf1"
server_key_version: "encryptionkey@0"
I20260811 15:08:49.065740 9414 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 15:08:49.068670 9583 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:49.070083 9414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 15:08:49.070226 9414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/wal
uuid: "660b3ae52063414f8fbd6024f6b8296a"
format_stamp: "Formatted at 2026-08-11 15:08:49 on dist-test-slave-1vmg"
server_key: "fd61b9ed12f8bdd40b2d26213f5a3910"
server_key_iv: "fb83dcaab09249abb20ddf45ac3efbf1"
server_key_version: "encryptionkey@0"
I20260811 15:08:49.070348 9414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:49.093372 9414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:49.096971 9414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:49.097177 9414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:49.106446 9414 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.60:33179
I20260811 15:08:49.106725 9667 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.60:33179 every 8 connection(s)
I20260811 15:08:49.107793 9414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/data/info.pb
I20260811 15:08:49.111057 9671 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:49.115480 9671 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } has no permanent_uuid. Determining permanent_uuid...
I20260811 15:08:49.117059 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 9414
Aug 11 15:08:49 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460929, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.60@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM
Aug 11 15:08:49 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.8.60@KRBTEST.COM
I20260811 15:08:49.131145 8520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.117891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41424 (local address 127.30.8.62:36293)
0811 15:08:49.118067 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.118072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.118958 (+ 886us) server_negotiation.cc:407] Connection header received
0811 15:08:49.119998 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.120002 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.120057 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.120154 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.121315 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.122148 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.123645 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.123995 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.124112 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.127098 (+ 2986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:49.127101 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:49.127103 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:49.127136 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:49.128826 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.129348 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.129353 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.129355 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.129453 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.129845 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.129849 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.129851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.130109 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:49.130214 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:49.130766 (+ 552us) server_negotiation.cc:299] Negotiation successful
0811 15:08:49.130908 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 15:08:49.132196 9677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.119114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50482 (local address 127.30.8.60:33179)
0811 15:08:49.119416 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.119425 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.119460 (+ 35us) server_negotiation.cc:407] Connection header received
0811 15:08:49.119771 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.119788 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.120020 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.120206 (+ 186us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.121198 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.122436 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.123645 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.124263 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.124346 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.127454 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:49.127460 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:49.127474 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:49.127497 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:49.129048 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.129561 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.129567 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.129568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.129721 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.130005 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.130008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.130009 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.130465 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:49.130679 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:49.130902 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 15:08:49.131291 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":183,"server-negotiator.queue_time_us":209,"spinlock_wait_cycles":768,"thread_start_us":110,"threads_started":1}
I20260811 15:08:49.132764 9673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.118216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.0.0.1:41424)
0811 15:08:49.118768 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:49.118815 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:49.119772 (+ 957us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:49.120231 (+ 459us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:49.120238 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:49.120715 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:49.121132 (+ 417us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.121156 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.122286 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.122289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.123522 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.123534 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.124161 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.124164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.124298 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.125004 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:49.125031 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:49.126879 (+ 1848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:49.128945 (+ 2066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.128956 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.128970 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.129237 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.129557 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.129561 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.129563 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.129727 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.130233 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:49.130241 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:49.130499 (+ 258us) client_negotiation.cc:769] Sending connection context
0811 15:08:49.130677 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 15:08:49.131114 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"spinlock_wait_cycles":2816,"thread_start_us":148,"threads_started":1}
I20260811 15:08:49.134390 9671 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 15:08:49 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460929, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.60@KRBTEST.COM for oryx/127.30.8.61@KRBTEST.COM
I20260811 15:08:49.142817 9673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.134965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:40131 (local address 127.0.0.1:52384)
0811 15:08:49.135131 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:49.135149 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:49.135242 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:49.135598 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:49.135601 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:49.135780 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:49.135901 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.135908 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.136776 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.136778 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.137334 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.137342 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.137749 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.137752 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.137837 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.138396 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:49.138415 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:49.139768 (+ 1353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:49.141452 (+ 1684us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.141455 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.141457 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.141677 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.141917 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.141919 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.141920 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.142006 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.142351 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:49.142353 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:49.142448 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 15:08:49.142537 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 15:08:49.142670 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":76}
I20260811 15:08:49.142894 9678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.135093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52384 (local address 127.30.8.61:40131)
0811 15:08:49.135382 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.135386 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.135399 (+ 13us) server_negotiation.cc:407] Connection header received
0811 15:08:49.135440 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.135442 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.135482 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.135565 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.136007 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.136688 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.137426 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.137660 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.137734 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.139932 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:49.139936 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:49.139938 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:49.139964 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:49.141328 (+ 1364us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.141765 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.141767 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.141769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.141843 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.142077 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.142079 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.142081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.142276 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:49.142351 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:49.142583 (+ 232us) server_negotiation.cc:299] Negotiation successful
0811 15:08:49.142695 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":130,"threads_started":1}
I20260811 15:08:49.143885 9671 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 15:08:49 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460929, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.60@KRBTEST.COM for oryx/127.30.8.60@KRBTEST.COM
I20260811 15:08:49.151568 9677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.144462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50496 (local address 127.30.8.60:33179)
0811 15:08:49.144554 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.144559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.144611 (+ 52us) server_negotiation.cc:407] Connection header received
0811 15:08:49.144743 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.144747 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.144784 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.144847 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.145248 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.145785 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.146516 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.146746 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.146833 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.148804 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:49.148808 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:49.148810 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:49.148831 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:49.149980 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.150370 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.150374 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.150375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.150447 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.150702 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.150705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.150707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.150897 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:49.151017 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:49.151266 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 15:08:49.151414 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":29}
I20260811 15:08:49.151568 9673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.144404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:33179 (local address 127.0.0.1:50496)
0811 15:08:49.144554 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:49.144570 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:49.144658 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:49.144872 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:49.144874 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:49.145062 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:49.145140 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.145145 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.145889 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.145893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.146396 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.146403 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.146847 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.146850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.146973 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.147430 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:49.147440 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:49.148667 (+ 1227us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:49.150087 (+ 1420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.150090 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.150093 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.150277 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.150534 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.150537 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.150539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.150618 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.151019 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:49.151022 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:49.151115 (+ 93us) client_negotiation.cc:769] Sending connection context
0811 15:08:49.151201 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 15:08:49.151360 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60,"mutex_wait_us":2}
I20260811 15:08:49.155308 9671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a: Bootstrap starting.
I20260811 15:08:49.157886 9671 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:49.158708 9671 log.cc:824] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:49.161667 9671 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a: No bootstrap required, opened a new log
I20260811 15:08:49.165098 9671 raft_consensus.cc:348] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } }
I20260811 15:08:49.165370 9671 raft_consensus.cc:374] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:49.165411 9671 raft_consensus.cc:729] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 660b3ae52063414f8fbd6024f6b8296a, State: Initialized, Role: FOLLOWER
I20260811 15:08:49.166055 9671 consensus_queue.cc:260] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } }
I20260811 15:08:49.166690 9682 sys_catalog.cc:455] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } } }
I20260811 15:08:49.166831 9682 sys_catalog.cc:458] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:08:49.167572 9671 sys_catalog.cc:564] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [sys.catalog]: configured and running, proceeding with master startup.
W20260811 15:08:49.173316 9699 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:08:49.173440 9699 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:08:49.177517 9414 master_runner.cc:428] Detected that this master 660b3ae52063414f8fbd6024f6b8296a is joining an existing cluster
I20260811 15:08:49.177604 9414 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.8.60:33179
I20260811 15:08:49.185890 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 8558
I20260811 15:08:49.221093 9677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.180936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50510 (local address 127.30.8.60:33179)
0811 15:08:49.181082 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.181087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.181498 (+ 411us) server_negotiation.cc:407] Connection header received
0811 15:08:49.181632 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.181635 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.181672 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.181721 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.182637 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.183391 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.184621 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.184978 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.185067 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.216598 (+ 31531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:49.216604 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:49.216608 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:49.216644 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:49.218591 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.219187 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.219191 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.219193 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.219284 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.219590 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.219593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.219595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.219912 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:49.220004 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:49.220720 (+ 716us) server_negotiation.cc:299] Negotiation successful
0811 15:08:49.220885 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":46,"server-negotiator.queue_time_us":69}
I20260811 15:08:49.223748 9711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.181075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:33179 (local address 127.0.0.1:50510)
0811 15:08:49.181415 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:49.181428 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:49.181542 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:49.181907 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:49.181910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:49.182262 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:49.182500 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.182510 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.183603 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.183606 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.184443 (+ 837us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.184454 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.185610 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.185614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.185809 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.214772 (+ 28963us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:49.214795 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:49.215446 (+ 651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:49.218757 (+ 3311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.218761 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.218763 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.219067 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.219393 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.219396 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.219399 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.219495 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.220344 (+ 849us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:49.220347 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:49.220464 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 15:08:49.220868 (+ 404us) client_negotiation.cc:240] Negotiation successful
0811 15:08:49.220998 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":109,"spinlock_wait_cycles":4352,"thread_start_us":96,"threads_started":1}
I20260811 15:08:49.227227 8520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.180704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41440 (local address 127.30.8.62:36293)
0811 15:08:49.180827 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.180831 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.181432 (+ 601us) server_negotiation.cc:407] Connection header received
0811 15:08:49.181633 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.181637 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.181686 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.181740 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.183082 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.183819 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.185031 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.185329 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.185436 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.222790 (+ 37354us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:49.222796 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:49.222799 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:49.222840 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:49.224916 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.225607 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.225612 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.225614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.225723 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.226093 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.226097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.226099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.226358 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:49.226438 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:49.226850 (+ 412us) server_negotiation.cc:299] Negotiation successful
0811 15:08:49.227036 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 15:08:49.227608 9678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.180421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52394 (local address 127.30.8.61:40131)
0811 15:08:49.180583 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.180588 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.180822 (+ 234us) server_negotiation.cc:407] Connection header received
0811 15:08:49.181159 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.181163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.181220 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.181306 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.182078 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.182909 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.184153 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.184519 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.184599 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.221259 (+ 36660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:49.221264 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:49.221268 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:49.221311 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:49.223389 (+ 2078us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.224253 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.224258 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.224260 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.224382 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.224802 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.224806 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.224808 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.225096 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:49.225180 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:49.227248 (+ 2068us) server_negotiation.cc:299] Negotiation successful
0811 15:08:49.227426 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 15:08:49.228155 9709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.180233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:40131 (local address 127.0.0.1:52394)
0811 15:08:49.180732 (+ 499us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:49.180749 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:49.180912 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:49.181443 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:49.181446 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:49.181703 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:49.181916 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.181925 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.183097 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.183100 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.184010 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.184021 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.184794 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.184798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.184909 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.220082 (+ 35173us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:49.220100 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:49.220561 (+ 461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:49.223588 (+ 3027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.223592 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.223594 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.224105 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.224543 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.224546 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.224548 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.224667 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.226831 (+ 2164us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:49.226834 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:49.227021 (+ 187us) client_negotiation.cc:769] Sending connection context
0811 15:08:49.227755 (+ 734us) client_negotiation.cc:240] Negotiation successful
0811 15:08:49.227945 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":394,"mutex_wait_us":46,"thread_start_us":132,"threads_started":1}
I20260811 15:08:49.228471 9712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.180233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.0.0.1:41440)
0811 15:08:49.181334 (+ 1101us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:49.181349 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:49.181509 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:49.181893 (+ 384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:49.181896 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:49.182258 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:49.182877 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.182889 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.184017 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.184021 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.184880 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.184889 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.185627 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.185631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.185748 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.220746 (+ 34998us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:49.220760 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:49.222557 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:49.225204 (+ 2647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.225208 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.225210 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.225465 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.225856 (+ 391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.225858 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.225860 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.225952 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.226518 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:49.226521 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:49.226632 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 15:08:49.227896 (+ 1264us) client_negotiation.cc:240] Negotiation successful
0811 15:08:49.228039 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1032,"mutex_wait_us":83,"thread_start_us":93,"threads_started":1}
I20260811 15:08:49.243547 9001 heartbeater.cc:503] Master 127.30.8.62:36293 was elected leader, sending a full tablet report...
I20260811 15:08:49.244573 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:37731
--local_ip_for_outbound_sockets=127.30.8.1
--tserver_master_addrs=127.30.8.62:36293
--webserver_port=37955
--webserver_interface=127.30.8.1
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.1.keytab
--principal=oryx/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:38255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.8.62:36293,127.30.8.61:40131
--tserver_master_addrs=127.30.8.62:36293,127.30.8.61:40131,127.30.8.60:33179 with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 15:08:49 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460927, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.62@KRBTEST.COM for oryx/127.30.8.60@KRBTEST.COM
I20260811 15:08:49.250891 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.240774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:33179 (local address 127.0.0.1:50524)
0811 15:08:49.240966 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:49.240980 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:49.241076 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:49.241391 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:49.241394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:49.241631 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:49.241755 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.241763 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.242757 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.242761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.243688 (+ 927us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.243703 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.244331 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.244336 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.244466 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.245496 (+ 1030us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:49.245511 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:49.247132 (+ 1621us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:49.249134 (+ 2002us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.249139 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.249141 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.249372 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.249702 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.249706 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.249708 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.249803 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.250291 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:49.250294 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:49.250401 (+ 107us) client_negotiation.cc:769] Sending connection context
0811 15:08:49.250504 (+ 103us) client_negotiation.cc:240] Negotiation successful
0811 15:08:49.250693 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107}
I20260811 15:08:49.250891 9677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.240923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50524 (local address 127.30.8.60:33179)
0811 15:08:49.241057 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.241062 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.241081 (+ 19us) server_negotiation.cc:407] Connection header received
0811 15:08:49.241206 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.241209 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.241272 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.241365 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.241896 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.242609 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.243870 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.244221 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.244322 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.247304 (+ 2982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:49.247308 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:49.247311 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:49.247342 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:49.248994 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.249498 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.249503 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.249505 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.249596 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.249932 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.249936 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.249937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.250169 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:49.250276 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:49.250588 (+ 312us) server_negotiation.cc:299] Negotiation successful
0811 15:08:49.250714 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 15:08:49.252537 8461 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.8.60:33179: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a"
member_type: NON_VOTER
last_known_addr {
host: "127.30.8.60"
port: 33179
}
attrs {
promote: true
}
}
dest_uuid: "c7378c939fdc4fd38b8eaab16d60d54f"
cas_config_opid_index: 5
I20260811 15:08:49.253222 8461 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: NON_VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: true } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: NON_VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: true } }
I20260811 15:08:49.254412 9391 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 660b3ae52063414f8fbd6024f6b8296a (127.30.8.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: NON_VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: true } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: NON_VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: true } } }
I20260811 15:08:49.256187 9391 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.8.60:33179
I20260811 15:08:49.256423 9391 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: NON_VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: true } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: NON_VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: true } } }
I20260811 15:08:49.256556 9391 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:49.257603 9623 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:49.259310 9623 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 1 FOLLOWER]: Refusing update from remote peer c7378c939fdc4fd38b8eaab16d60d54f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 15:08:49.260502 9414 master.cc:537] Master@127.30.8.60:33179 shutting down...
I20260811 15:08:49.260769 9678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.255019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52402 (local address 127.30.8.61:40131)
0811 15:08:49.255164 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.255169 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.255679 (+ 510us) server_negotiation.cc:407] Connection header received
0811 15:08:49.255908 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.255912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.256006 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.256102 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:49.256584 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.257394 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.259017 (+ 1623us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.259653 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.259788 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.259991 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:49.260518 (+ 527us) server_negotiation.cc:299] Negotiation successful
0811 15:08:49.260626 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 15:08:49.261111 9397 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [LEADER]: Connected to new peer: Peer: permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: NON_VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 15:08:49.263214 8440 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260811 15:08:49.264925 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.254674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:40131 (local address 127.0.0.1:52402)
0811 15:08:49.255593 (+ 919us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:49.255610 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:49.255763 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:49.256294 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:49.256297 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:49.256317 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:49.256426 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.256434 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.257567 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.257572 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.258806 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.258820 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.260212 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.260216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.260318 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.260372 (+ 54us) client_negotiation.cc:769] Sending connection context
0811 15:08:49.264628 (+ 4256us) client_negotiation.cc:240] Negotiation successful
0811 15:08:49.264760 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":825}
I20260811 15:08:49.265631 9486 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 1 LEARNER]: Refusing update from remote peer c7378c939fdc4fd38b8eaab16d60d54f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 15:08:49.266198 9397 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: NON_VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260811 15:08:49.267093 8440 consensus_peers.cc:487] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f -> Peer 660b3ae52063414f8fbd6024f6b8296a (127.30.8.60:33179): Couldn't send request to peer 660b3ae52063414f8fbd6024f6b8296a. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 15:08:49.267867 9486 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 660b3ae52063414f8fbd6024f6b8296a (127.30.8.60) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: NON_VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: true } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: NON_VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: true } } }
I20260811 15:08:49.270422 9543 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7378c939fdc4fd38b8eaab16d60d54f. Latest consensus state: current_term: 1 leader_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: NON_VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: true } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: NON_VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: true } } }
I20260811 15:08:49.270591 9543 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: This master's current role is: LEARNER
I20260811 15:08:49.270684 9543 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: NON_VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: true } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: NON_VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: true } } }
I20260811 15:08:49.270753 9543 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: This master's current role is: LEARNER
I20260811 15:08:49.271450 9414 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:49.271981 9414 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:49.272140 9414 tablet_replica.cc:330] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a: stopping tablet replica
I20260811 15:08:49.272672 9397 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f: attempting to promote NON_VOTER dd9736da73324408b58ba279deb3f85e to VOTER
I20260811 15:08:49.273316 9397 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: NON_VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: true } }
I20260811 15:08:49.276727 9486 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 1 LEARNER]: Refusing update from remote peer c7378c939fdc4fd38b8eaab16d60d54f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 15:08:49.277779 9397 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 15:08:49.280679 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.280257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:33179 (local address 127.0.0.1:50534)
0811 15:08:49.280464 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:49.280569 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:33179: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
W20260811 15:08:49.281098 8440 consensus_peers.cc:487] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f -> Peer 660b3ae52063414f8fbd6024f6b8296a (127.30.8.60:33179): Couldn't send request to peer 660b3ae52063414f8fbd6024f6b8296a. Status: Network error: Client connection negotiation failed: client connection to 127.30.8.60:33179: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 15:08:49.281082 9391 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, dd9736da73324408b58ba279deb3f85e (127.30.8.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: NON_VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: true } } }
I20260811 15:08:49.283036 9391 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: NON_VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: true } } }
I20260811 15:08:49.283310 9391 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:49.283772 9398 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: NON_VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: true } } }
I20260811 15:08:49.283907 9398 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:49.284641 9486 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, dd9736da73324408b58ba279deb3f85e (127.30.8.61) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: NON_VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: true } } }
I20260811 15:08:49.286988 9716 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: NON_VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: true } } }
I20260811 15:08:49.287142 9716 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:08:49.291505 9414 master.cc:552] Master@127.30.8.60:33179 shutdown complete.
I20260811 15:08:49.291584 9414 master_runner.cc:305] Clearing existing system tablet
I20260811 15:08:49.291847 9414 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:49.291911 9414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:49.293375 9414 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 15:08:49.294322 9721 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:49.294533 9414 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:49.294667 9414 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:49.294710 9414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/wal
uuid: "660b3ae52063414f8fbd6024f6b8296a"
format_stamp: "Formatted at 2026-08-11 15:08:49 on dist-test-slave-1vmg"
server_key: "fd61b9ed12f8bdd40b2d26213f5a3910"
server_key_iv: "fb83dcaab09249abb20ddf45ac3efbf1"
server_key_version: "encryptionkey@0"
I20260811 15:08:49.296590 9414 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:49.298733 9414 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 15:08:49.298834 9414 log.cc:1196] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 15:08:49.299151 9414 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a: Deleting consensus metadata
I20260811 15:08:49.299342 9414 master_runner.cc:315] Copying system tablet from 127.30.8.62:36293
I20260811 15:08:49.300611 9414 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a: tablet copy: Beginning tablet copy session from remote peer at address 127.30.8.62:36293
I20260811 15:08:49.310243 8520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.301350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41448 (local address 127.30.8.62:36293)
0811 15:08:49.301519 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.301525 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.301788 (+ 263us) server_negotiation.cc:407] Connection header received
0811 15:08:49.301925 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.301929 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.301987 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.302069 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.302574 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.303205 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.304111 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.304341 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.304446 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.306247 (+ 1801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:49.306251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:49.306254 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:49.306291 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:49.308268 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.308816 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.308821 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.308823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.308927 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.309226 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.309229 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.309231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.309484 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:49.309581 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:49.309898 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 15:08:49.310032 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 15:08:49.310267 9733 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.301240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.0.0.1:41448)
0811 15:08:49.301674 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:49.301688 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:49.301817 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:49.302105 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:49.302108 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:49.302324 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:49.302426 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.302433 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.303329 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.303334 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.303950 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.303961 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.304490 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.304496 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.304592 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.305163 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:49.305178 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:49.305948 (+ 770us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:49.308430 (+ 2482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.308434 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.308437 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.308698 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.309029 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.309032 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.309035 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.309126 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.309592 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:49.309596 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:49.309712 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 15:08:49.309808 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 15:08:49.310031 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":102,"threads_started":1}
I20260811 15:08:49.311292 8493 tablet_copy_service.cc:140] P c7378c939fdc4fd38b8eaab16d60d54f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 660b3ae52063414f8fbd6024f6b8296a ({username='oryx', principal='oryx/127.30.8.60@KRBTEST.COM'} at 127.0.0.1:41448)
I20260811 15:08:49.311408 8493 tablet_copy_service.cc:161] P c7378c939fdc4fd38b8eaab16d60d54f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 660b3ae52063414f8fbd6024f6b8296a at {username='oryx', principal='oryx/127.30.8.60@KRBTEST.COM'} at 127.0.0.1:41448: session id = 660b3ae52063414f8fbd6024f6b8296a-00000000000000000000000000000000
I20260811 15:08:49.312532 8493 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 15:08:49.314182 9414 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 15:08:49.316747 9414 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a: tablet copy: Starting download of 0 data blocks...
I20260811 15:08:49.316984 9414 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a: tablet copy: Starting download of 1 WAL segments...
I20260811 15:08:49.318783 9414 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 15:08:49.320919 8493 tablet_copy_service.cc:342] P c7378c939fdc4fd38b8eaab16d60d54f: Request end of tablet copy session 660b3ae52063414f8fbd6024f6b8296a-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.8.60@KRBTEST.COM'} at 127.0.0.1:41448
I20260811 15:08:49.321022 8493 tablet_copy_service.cc:433] P c7378c939fdc4fd38b8eaab16d60d54f: ending tablet copy session 660b3ae52063414f8fbd6024f6b8296a-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 660b3ae52063414f8fbd6024f6b8296a
I20260811 15:08:49.334666 9414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:49.337376 9739 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:49.338847 9735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:49.348057 9734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:49.348491 9414 server_base.cc:1034] running on GCE node
I20260811 15:08:49.348886 9414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:49.349325 9414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:49.350471 9414 hybrid_clock.cc:648] HybridClock initialized: now 1786460929350458 us; error 39 us; skew 500 ppm
Aug 11 15:08:49 dist-test-slave-1vmg krb5kdc[8381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460929, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:49.352545 9414 init.cc:377] Logged in from keytab as oryx/127.30.8.60@KRBTEST.COM (short username oryx)
I20260811 15:08:49.353163 9414 webserver.cc:460] Webserver started at http://127.30.8.60:44103/ using document root <none> and password file <none>
I20260811 15:08:49.353371 9414 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:49.353405 9414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:49.354950 9414 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:49.356092 9749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:49.356372 9414 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:49.356453 9414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/wal
uuid: "660b3ae52063414f8fbd6024f6b8296a"
format_stamp: "Formatted at 2026-08-11 15:08:49 on dist-test-slave-1vmg"
server_key: "fd61b9ed12f8bdd40b2d26213f5a3910"
server_key_iv: "fb83dcaab09249abb20ddf45ac3efbf1"
server_key_version: "encryptionkey@0"
I20260811 15:08:49.356555 9414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:49.370908 9414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:49.373099 9414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:49.378693 9414 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.60:33179
I20260811 15:08:49.378773 9828 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.60:33179 every 8 connection(s)
I20260811 15:08:49.379675 9414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/master-2/data/info.pb
I20260811 15:08:49.381810 9829 sys_catalog.cc:263] Verifying existing consensus state
I20260811 15:08:49.383054 9829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a: Bootstrap starting.
I20260811 15:08:49.394209 9829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 15:08:49.394594 9829 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a: Bootstrap complete.
I20260811 15:08:49.395340 9829 raft_consensus.cc:348] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: NON_VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: true } }
I20260811 15:08:49.395498 9829 raft_consensus.cc:729] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 660b3ae52063414f8fbd6024f6b8296a, State: Initialized, Role: LEARNER
I20260811 15:08:49.395605 9829 consensus_queue.cc:260] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: NON_VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: true } }
I20260811 15:08:49.396034 9833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: NON_VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: true } } }
I20260811 15:08:49.396173 9833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [sys.catalog]: This master's current role is: LEARNER
I20260811 15:08:49.396198 9829 sys_catalog.cc:564] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:08:49.400532 9847 catalog_manager.cc:1259] Loaded cluster ID: 035c4a9125c04bba9b2b98a3ebcd918c
I20260811 15:08:49.400614 9847 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a: loading cluster ID for follower catalog manager: success
I20260811 15:08:49.402439 9847 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a: acquiring CA information for follower catalog manager: success
I20260811 15:08:49.403931 9847 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 15:08:49.409642 9713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:49.409958 9713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:49.410032 9713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:49.414340 9713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:49.414422 9713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:49.414510 9713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:08:49.420666 9713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:38255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.1.keytab
--principal=oryx/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:37731
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=37955
--tserver_master_addrs=127.30.8.62:36293,127.30.8.61:40131,127.30.8.60:33179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:49.422003 9713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:49.423250 9713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:49.433948 9858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:49.433970 9863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:49.433967 9856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:49.434237 9713 server_base.cc:1034] running on GCE node
I20260811 15:08:49.434873 9713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:49.435663 9713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:49.436846 9713 hybrid_clock.cc:648] HybridClock initialized: now 1786460929436825 us; error 38 us; skew 500 ppm
Aug 11 15:08:49 dist-test-slave-1vmg krb5kdc[8381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460929, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:49.439987 9713 init.cc:377] Logged in from keytab as oryx/127.30.8.1@KRBTEST.COM (short username oryx)
I20260811 15:08:49.441064 9713 webserver.cc:460] Webserver started at http://127.30.8.1:37955/ using document root <none> and password file <none>
I20260811 15:08:49.441700 9713 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:49.441802 9713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:49.446733 9713 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:08:49.449443 9871 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:49.450884 9713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:08:49.451102 9713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "8b439d6b462a49188ab751501b266fdd"
format_stamp: "Formatted at 2026-08-11 15:08:47 on dist-test-slave-1vmg"
server_key: "c49d8c5eab4caac923213147c28953c9"
server_key_iv: "55e9a590e619c76204df8713862adba0"
server_key_version: "encryptionkey@0"
I20260811 15:08:49.451727 9713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:49.469444 9713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:49.472658 9713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:49.472878 9713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:49.473565 9713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:49.474566 9713 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:49.474619 9713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:49.474680 9713 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:49.474711 9713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:49.485208 9713 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:37731
I20260811 15:08:49.485222 10037 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:37731 every 8 connection(s)
I20260811 15:08:49.486526 9713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:08:49.495405 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 9713
I20260811 15:08:49.495627 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 8784
Aug 11 15:08:49 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460929, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.60@KRBTEST.COM
Aug 11 15:08:49 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460929, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM
I20260811 15:08:49.504762 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.488657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:40883 (local address 127.30.8.60:33179)
0811 15:08:49.488990 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.488995 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.490506 (+ 1511us) server_negotiation.cc:407] Connection header received
0811 15:08:49.491746 (+ 1240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.491750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.491811 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.491871 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.493988 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.495058 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.496342 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.496642 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.496706 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.500400 (+ 3694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:49.500404 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:49.500407 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:49.500443 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:49.502050 (+ 1607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.502613 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.502618 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.502620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.502719 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.503156 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.503159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.503161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.503411 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:49.503518 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:49.504390 (+ 872us) server_negotiation.cc:299] Negotiation successful
0811 15:08:49.504552 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":253,"thread_start_us":104,"threads_started":1}
I20260811 15:08:49.506300 10052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.489103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:33179 (local address 127.30.8.1:40883)
0811 15:08:49.490320 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:49.490382 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:49.491479 (+ 1097us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:49.492312 (+ 833us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:49.492322 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:49.493027 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:49.493793 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.493816 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.495193 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.495196 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.496195 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.496206 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.496938 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.496942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.497227 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.497913 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:49.497986 (+ 73us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:49.500168 (+ 2182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:49.502182 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.502189 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.502211 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.502496 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.502820 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.502824 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.502826 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.503054 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.503561 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:49.503571 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:49.504038 (+ 467us) client_negotiation.cc:769] Sending connection context
0811 15:08:49.504337 (+ 299us) client_negotiation.cc:240] Negotiation successful
0811 15:08:49.504821 (+ 484us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1121,"mutex_wait_us":119,"spinlock_wait_cycles":28544,"thread_start_us":122,"threads_started":1}
I20260811 15:08:49.506711 8520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.490303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:36453 (local address 127.30.8.62:36293)
0811 15:08:49.490438 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.490443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.491217 (+ 774us) server_negotiation.cc:407] Connection header received
0811 15:08:49.491756 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.491761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.491818 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.491874 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.493985 (+ 2111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.494582 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.499276 (+ 4694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.499636 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.499721 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.503304 (+ 3583us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:49.503309 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:49.503311 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:49.503345 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:49.504874 (+ 1529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.505456 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.505459 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.505461 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.505547 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.505835 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.505839 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.505840 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.506032 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:49.506134 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:49.506414 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 15:08:49.506554 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 15:08:49.507381 10049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.489367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.1:36453)
0811 15:08:49.491106 (+ 1739us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:49.491121 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:49.491493 (+ 372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:49.492312 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:49.492322 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:49.493012 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:49.493794 (+ 782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.493815 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.494746 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.494748 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.499090 (+ 4342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.499105 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.499941 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.499945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.500044 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.500631 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:49.500646 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:49.503043 (+ 2397us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:49.505114 (+ 2071us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.505118 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.505120 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.505336 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.505649 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.505652 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.505654 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.505735 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.506136 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:49.506139 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:49.506235 (+ 96us) client_negotiation.cc:769] Sending connection context
0811 15:08:49.506313 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 15:08:49.507242 (+ 929us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1673,"mutex_wait_us":3094,"thread_start_us":79,"threads_started":1}
I20260811 15:08:49.508517 10045 heartbeater.cc:347] Connected to a master server at 127.30.8.62:36293
I20260811 15:08:49.508809 10045 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:49.509378 10045 heartbeater.cc:511] Master 127.30.8.62:36293 requested a full tablet report, sending...
I20260811 15:08:49.510805 8461 ts_manager.cc:194] Re-registered known tserver with Master: 8b439d6b462a49188ab751501b266fdd (127.30.8.1:37731)
I20260811 15:08:49.511466 8461 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:36453
I20260811 15:08:49.512494 9226 heartbeater.cc:503] Master 127.30.8.62:36293 was elected leader, sending a full tablet report...
I20260811 15:08:49.522500 10044 heartbeater.cc:347] Connected to a master server at 127.30.8.60:33179
I20260811 15:08:49.522637 10044 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:49.522954 10044 heartbeater.cc:511] Master 127.30.8.60:33179 requested a full tablet report, sending...
I20260811 15:08:49.524554 9773 ts_manager.cc:194] Registered new tserver with Master: 8b439d6b462a49188ab751501b266fdd (127.30.8.1:37731)
Aug 11 15:08:49 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460929, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.61@KRBTEST.COM
I20260811 15:08:49.528690 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:41249
--local_ip_for_outbound_sockets=127.30.8.2
--tserver_master_addrs=127.30.8.62:36293
--webserver_port=44135
--webserver_interface=127.30.8.2
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.2.keytab
--principal=oryx/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:38255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.8.62:36293,127.30.8.61:40131
--tserver_master_addrs=127.30.8.62:36293,127.30.8.61:40131,127.30.8.60:33179 with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:08:49.528899 9678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.488700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:45171 (local address 127.30.8.61:40131)
0811 15:08:49.488838 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.488842 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.490545 (+ 1703us) server_negotiation.cc:407] Connection header received
0811 15:08:49.491746 (+ 1201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.491750 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.491810 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.491869 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.493986 (+ 2117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.494832 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.521087 (+ 26255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.521448 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.521536 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.525447 (+ 3911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:49.525451 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:49.525454 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:49.525490 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:49.527030 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.527522 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.527526 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.527528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.527643 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.527900 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.527903 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.527904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.528159 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:49.528251 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:49.528614 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 15:08:49.528736 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 15:08:49.528919 10051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.489103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:40131 (local address 127.30.8.1:45171)
0811 15:08:49.490320 (+ 1217us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:49.490479 (+ 159us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:49.491482 (+ 1003us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:49.492312 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:49.492322 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:49.493030 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:49.493797 (+ 767us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.493815 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.495070 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.495073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.520888 (+ 25815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.520912 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.521780 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.521784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.522155 (+ 371us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.522831 (+ 676us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:49.522847 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:49.525264 (+ 2417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:49.527154 (+ 1890us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.527158 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.527159 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.527381 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.527730 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.527733 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.527734 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.527807 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.528311 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:49.528315 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:49.528429 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 15:08:49.528514 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 15:08:49.528715 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":992,"mutex_wait_us":140,"spinlock_wait_cycles":28288,"thread_start_us":137,"threads_started":1}
I20260811 15:08:49.529825 10046 heartbeater.cc:347] Connected to a master server at 127.30.8.61:40131
I20260811 15:08:49.529934 10046 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:49.530117 10046 heartbeater.cc:511] Master 127.30.8.61:40131 requested a full tablet report, sending...
I20260811 15:08:49.531419 9470 ts_manager.cc:194] Registered new tserver with Master: 8b439d6b462a49188ab751501b266fdd (127.30.8.1:37731)
W20260811 15:08:49.652302 10053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:49.652626 10053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:49.652685 10053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:49.657161 10053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:49.657249 10053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:49.657320 10053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:08:49.663230 10053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:38255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.2.keytab
--principal=oryx/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:41249
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=44135
--tserver_master_addrs=127.30.8.62:36293,127.30.8.61:40131,127.30.8.60:33179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:49.664520 10053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:49.665638 10053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:49.676422 10065 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:49.676445 10064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:49.676443 10068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:49.676808 10053 server_base.cc:1034] running on GCE node
I20260811 15:08:49.677289 10053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:49.678031 10053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:49.679199 10053 hybrid_clock.cc:648] HybridClock initialized: now 1786460929679182 us; error 31 us; skew 500 ppm
Aug 11 15:08:49 dist-test-slave-1vmg krb5kdc[8381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460929, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:49.682324 10053 init.cc:377] Logged in from keytab as oryx/127.30.8.2@KRBTEST.COM (short username oryx)
I20260811 15:08:49.683399 10053 webserver.cc:460] Webserver started at http://127.30.8.2:44135/ using document root <none> and password file <none>
I20260811 15:08:49.684031 10053 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:49.684132 10053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:49.688652 10053 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 15:08:49.691021 10078 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:49.692420 10053 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:08:49.692569 10053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "fea2d50e2519476984b8fec3e8d172bc"
format_stamp: "Formatted at 2026-08-11 15:08:48 on dist-test-slave-1vmg"
server_key: "f28adf8c2e65e5b4aeceab9e4a837e11"
server_key_iv: "fd04548ba8fc1b1d22f83cb475246db9"
server_key_version: "encryptionkey@0"
I20260811 15:08:49.693244 10053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:49.723620 10053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:49.726768 10053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:49.727010 10053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:49.727684 10053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:49.728660 10053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:49.728710 10053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:49.728767 10053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:49.728788 10053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:49.738473 10053 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:41249
I20260811 15:08:49.738519 10250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:41249 every 8 connection(s)
I20260811 15:08:49.739796 10053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:08:49.747609 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 10053
I20260811 15:08:49.747793 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 9007
Aug 11 15:08:49 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460929, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM
Aug 11 15:08:49 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460929, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.61@KRBTEST.COM
I20260811 15:08:49.758873 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.746230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:33179 (local address 127.0.0.1:50550)
0811 15:08:49.746338 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:49.746352 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:49.746421 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:49.746883 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:49.746886 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:49.746905 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:49.747024 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.747031 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.748016 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.748032 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.749516 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.749530 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.758352 (+ 8822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.758357 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.758487 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.758524 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 15:08:49.758589 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 15:08:49.758697 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260811 15:08:49.759866 10261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.746326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50550 (local address 127.30.8.60:33179)
0811 15:08:49.746618 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.746624 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.746637 (+ 13us) server_negotiation.cc:407] Connection header received
0811 15:08:49.746683 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.746686 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.746741 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.746862 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:49.747157 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.747886 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.757246 (+ 9360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.758161 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.759295 (+ 1134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.759523 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:49.759572 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 15:08:49.759673 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":105,"threads_started":1}
I20260811 15:08:49.760263 8520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.742296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:36845 (local address 127.30.8.62:36293)
0811 15:08:49.742532 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.742536 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.744740 (+ 2204us) server_negotiation.cc:407] Connection header received
0811 15:08:49.745851 (+ 1111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.745856 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.745908 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.745988 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.747565 (+ 1577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.748368 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.750372 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.750713 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.750796 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.755235 (+ 4439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:49.755240 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:49.755243 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:49.755280 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:49.757463 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.758118 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.758122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.758124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.758236 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.758738 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.758742 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.758743 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.759008 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:49.759103 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:49.759976 (+ 873us) server_negotiation.cc:299] Negotiation successful
0811 15:08:49.760109 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166}
I20260811 15:08:49.760273 9678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.744445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:56657 (local address 127.30.8.61:40131)
0811 15:08:49.744577 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.744582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.744605 (+ 23us) server_negotiation.cc:407] Connection header received
0811 15:08:49.745432 (+ 827us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.745437 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.745507 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.745603 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.747577 (+ 1974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.748380 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.750374 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.750717 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.750796 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.755242 (+ 4446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:49.755246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:49.755249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:49.755280 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:49.757155 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.758118 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.758122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.758124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.758240 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.758729 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.758733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.758734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.759025 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:49.759103 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:49.759976 (+ 873us) server_negotiation.cc:299] Negotiation successful
0811 15:08:49.760109 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 15:08:49.761303 9791 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 15:08:49.762079 10259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.742739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.2:36845)
0811 15:08:49.744595 (+ 1856us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:49.744611 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:49.745098 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:49.746134 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:49.746137 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:49.746526 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:49.747394 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.747416 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.748566 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.748569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.750224 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.750237 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.751057 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.751061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.751423 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.752318 (+ 895us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:49.752330 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:49.754955 (+ 2625us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:49.757630 (+ 2675us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.757632 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.757634 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.757987 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.758356 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.758359 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.758361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.758605 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.759187 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:49.759196 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:49.759601 (+ 405us) client_negotiation.cc:769] Sending connection context
0811 15:08:49.760435 (+ 834us) client_negotiation.cc:240] Negotiation successful
0811 15:08:49.760900 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1721,"mutex_wait_us":550,"spinlock_wait_cycles":52992,"thread_start_us":167,"threads_started":1}
I20260811 15:08:49.762038 9833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [sys.catalog]: SysCatalogTable state changed. Reason: New leader c7378c939fdc4fd38b8eaab16d60d54f. Latest consensus state: current_term: 1 leader_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: NON_VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: true } } }
I20260811 15:08:49.762199 9833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [sys.catalog]: This master's current role is: LEARNER
I20260811 15:08:49.763015 10260 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.742740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:40131 (local address 127.30.8.2:56657)
0811 15:08:49.743905 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:49.743970 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:49.745095 (+ 1125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:49.745849 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:49.745860 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:49.746667 (+ 807us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:49.747394 (+ 727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.747415 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.748649 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.748651 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.750231 (+ 1580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.750237 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.751057 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.751061 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.751427 (+ 366us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.752268 (+ 841us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:49.752296 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:49.754955 (+ 2659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:49.757462 (+ 2507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.757470 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.757495 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.757987 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.758356 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.758359 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.758361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.758605 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.759379 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:49.759382 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:49.759601 (+ 219us) client_negotiation.cc:769] Sending connection context
0811 15:08:49.761171 (+ 1570us) client_negotiation.cc:240] Negotiation successful
0811 15:08:49.761346 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":890,"mutex_wait_us":584,"spinlock_wait_cycles":26368,"thread_start_us":622,"threads_started":1}
I20260811 15:08:49.763784 10254 heartbeater.cc:347] Connected to a master server at 127.30.8.62:36293
I20260811 15:08:49.764019 10255 heartbeater.cc:347] Connected to a master server at 127.30.8.61:40131
I20260811 15:08:49.764070 10254 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:49.764088 10255 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:49.764752 10254 heartbeater.cc:511] Master 127.30.8.62:36293 requested a full tablet report, sending...
I20260811 15:08:49.765362 10255 heartbeater.cc:511] Master 127.30.8.61:40131 requested a full tablet report, sending...
I20260811 15:08:49.766040 8461 ts_manager.cc:194] Re-registered known tserver with Master: fea2d50e2519476984b8fec3e8d172bc (127.30.8.2:41249)
I20260811 15:08:49.766108 9470 ts_manager.cc:194] Registered new tserver with Master: fea2d50e2519476984b8fec3e8d172bc (127.30.8.2:41249)
I20260811 15:08:49.766678 8461 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:36845
Aug 11 15:08:49 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460929, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.60@KRBTEST.COM
I20260811 15:08:49.780369 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:45381
--local_ip_for_outbound_sockets=127.30.8.3
--tserver_master_addrs=127.30.8.62:36293
--webserver_port=34479
--webserver_interface=127.30.8.3
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.3.keytab
--principal=oryx/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:38255
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.8.62:36293,127.30.8.61:40131
--tserver_master_addrs=127.30.8.62:36293,127.30.8.61:40131,127.30.8.60:33179 with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:08:49.780892 10258 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.742739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:33179 (local address 127.30.8.2:52069)
0811 15:08:49.744449 (+ 1710us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:49.744468 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:49.745148 (+ 680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:49.746066 (+ 918us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:49.746069 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:49.746676 (+ 607us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:49.771439 (+ 24763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.771460 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.772844 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.772848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.773712 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.773727 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.774841 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.774850 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.774988 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.775545 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:49.775560 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:49.777197 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:49.779026 (+ 1829us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.779030 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.779032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.779319 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.779615 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.779618 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.779620 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.779716 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.780271 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:49.780274 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:49.780390 (+ 116us) client_negotiation.cc:769] Sending connection context
0811 15:08:49.780498 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 15:08:49.780692 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1565,"mutex_wait_us":47,"spinlock_wait_cycles":8448,"thread_start_us":215,"threads_started":1}
I20260811 15:08:49.780930 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.742141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:52069 (local address 127.30.8.60:33179)
0811 15:08:49.742441 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.742446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.744757 (+ 2311us) server_negotiation.cc:407] Connection header received
0811 15:08:49.745543 (+ 786us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.745547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.745617 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.745700 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.771659 (+ 25959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.772689 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.773845 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.774088 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.774173 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.777391 (+ 3218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:49.777395 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:49.777397 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:49.777428 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:49.778858 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.779425 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.779427 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.779429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.779510 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.779841 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.779844 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.779846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.780146 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:49.780253 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:49.780601 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 15:08:49.780756 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203}
I20260811 15:08:49.782696 10252 heartbeater.cc:347] Connected to a master server at 127.30.8.60:33179
I20260811 15:08:49.782817 10252 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:49.783102 10252 heartbeater.cc:511] Master 127.30.8.60:33179 requested a full tablet report, sending...
I20260811 15:08:49.783885 9773 ts_manager.cc:194] Registered new tserver with Master: fea2d50e2519476984b8fec3e8d172bc (127.30.8.2:41249)
W20260811 15:08:49.905579 10262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:49.905913 10262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:49.905958 10262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:49.910355 10262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:49.910444 10262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:49.910509 10262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:08:49.916311 10262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:38255
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.3.keytab
--principal=oryx/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:45381
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=34479
--tserver_master_addrs=127.30.8.62:36293,127.30.8.61:40131,127.30.8.60:33179
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:49.917537 10262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:49.918644 10262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:49.929221 10282 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:49.929203 10278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:49.929206 10277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:49.929445 10262 server_base.cc:1034] running on GCE node
I20260811 15:08:49.930135 10262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:49.930876 10262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:49.932140 10262 hybrid_clock.cc:648] HybridClock initialized: now 1786460929932107 us; error 44 us; skew 500 ppm
Aug 11 15:08:49 dist-test-slave-1vmg krb5kdc[8381](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460929, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:49.935171 10262 init.cc:377] Logged in from keytab as oryx/127.30.8.3@KRBTEST.COM (short username oryx)
I20260811 15:08:49.936229 10262 webserver.cc:460] Webserver started at http://127.30.8.3:34479/ using document root <none> and password file <none>
I20260811 15:08:49.936851 10262 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:49.936968 10262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:49.941433 10262 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 15:08:49.943789 10289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:49.945166 10262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 15:08:49.945313 10262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "0f34c101cebf4318a7484b40cdfc072a"
format_stamp: "Formatted at 2026-08-11 15:08:48 on dist-test-slave-1vmg"
server_key: "4a4237d6e2f04f770ac66096ed1a3d40"
server_key_iv: "39fb3a37a23e2e762dc8e99d5a5645ff"
server_key_version: "encryptionkey@0"
I20260811 15:08:49.945901 10262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:49.963341 10262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:49.966548 10262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:49.966758 10262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:49.967519 10262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:08:49.968500 10262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:49.968554 10262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:49.968611 10262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:49.968641 10262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:49.978319 10262 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:45381
I20260811 15:08:49.978350 10474 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:45381 every 8 connection(s)
I20260811 15:08:49.979591 10262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:08:49.989907 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 10262
Aug 11 15:08:49 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460929, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.61@KRBTEST.COM
Aug 11 15:08:49 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460929, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM
I20260811 15:08:49.998128 9678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.983395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:37311 (local address 127.30.8.61:40131)
0811 15:08:49.983516 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.983520 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.983696 (+ 176us) server_negotiation.cc:407] Connection header received
0811 15:08:49.984946 (+ 1250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.984949 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.985000 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.985072 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.986715 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.987582 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.989078 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.989415 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.989507 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.993722 (+ 4215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:49.993726 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:49.993729 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:49.993755 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:49.995472 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.996186 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.996191 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.996192 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.996286 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:49.996706 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:49.996709 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:49.996711 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:49.996945 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:49.997034 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:49.997807 (+ 773us) server_negotiation.cc:299] Negotiation successful
0811 15:08:49.997944 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 15:08:50.000136 10484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.982179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:40131 (local address 127.30.8.3:37311)
0811 15:08:49.983548 (+ 1369us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:49.983561 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:49.984432 (+ 871us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:49.985187 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:49.985190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:49.985721 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:49.986544 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.986559 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.987696 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.987699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.988923 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.988934 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.989597 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.989601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.989863 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.990663 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:49.990689 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:49.993494 (+ 2805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:49.995617 (+ 2123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.995625 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.995647 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.996047 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.996393 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:49.996396 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:49.996398 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:49.996591 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:49.997101 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:49.997108 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:49.997494 (+ 386us) client_negotiation.cc:769] Sending connection context
0811 15:08:49.998381 (+ 887us) client_negotiation.cc:240] Negotiation successful
0811 15:08:49.998801 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1071,"mutex_wait_us":316,"spinlock_wait_cycles":30464,"thread_start_us":157,"threads_started":1}
I20260811 15:08:50.001691 10477 heartbeater.cc:347] Connected to a master server at 127.30.8.61:40131
I20260811 15:08:50.001986 10477 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:50.002635 10477 heartbeater.cc:511] Master 127.30.8.61:40131 requested a full tablet report, sending...
I20260811 15:08:50.003468 8520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.982468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:45097 (local address 127.30.8.62:36293)
0811 15:08:49.982715 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.982720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.983656 (+ 936us) server_negotiation.cc:407] Connection header received
0811 15:08:49.984644 (+ 988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.984647 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.984711 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.984800 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.986712 (+ 1912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.987430 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.989080 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.989325 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.989387 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.999319 (+ 9932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:49.999324 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:49.999327 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:49.999357 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:50.001240 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:50.001934 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:50.001937 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:50.001939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:50.002035 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:50.002388 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:50.002392 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:50.002394 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:50.002642 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:50.002745 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:50.003182 (+ 437us) server_negotiation.cc:299] Negotiation successful
0811 15:08:50.003316 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162}
I20260811 15:08:50.004036 9470 ts_manager.cc:194] Registered new tserver with Master: 0f34c101cebf4318a7484b40cdfc072a (127.30.8.3:45381)
I20260811 15:08:50.004417 10480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.982179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.3:45097)
0811 15:08:49.983478 (+ 1299us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:49.983534 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:49.984406 (+ 872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:49.985005 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:49.985015 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:49.985691 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:49.986541 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.986561 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.987593 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.987597 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.988955 (+ 1358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.988962 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.989597 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.989601 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.989865 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.990813 (+ 948us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:49.990826 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:49.999182 (+ 8356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:50.001499 (+ 2317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:50.001504 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:50.001506 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:50.001802 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:50.002159 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:50.002162 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:50.002164 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:50.002261 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:50.002848 (+ 587us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:50.002852 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:50.002983 (+ 131us) client_negotiation.cc:769] Sending connection context
0811 15:08:50.003984 (+ 1001us) client_negotiation.cc:240] Negotiation successful
0811 15:08:50.004215 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1175,"mutex_wait_us":375,"spinlock_wait_cycles":73088,"thread_start_us":129,"threads_started":1}
Aug 11 15:08:49 dist-test-slave-1vmg krb5kdc[8381](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460929, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.60@KRBTEST.COM
I20260811 15:08:50.005460 10261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.993736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50564 (local address 127.30.8.60:33179)
0811 15:08:49.993858 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.993863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.994784 (+ 921us) server_negotiation.cc:407] Connection header received
0811 15:08:49.995014 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.995018 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.995087 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.995145 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.996120 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.996902 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.998052 (+ 1150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.998393 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.998459 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:50.001191 (+ 2732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:50.001195 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:50.001198 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:50.001229 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:50.003167 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:50.003757 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:50.003761 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:50.003763 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:50.003859 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:50.004168 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:50.004171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:50.004173 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:50.004420 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:50.004535 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:50.005147 (+ 612us) server_negotiation.cc:299] Negotiation successful
0811 15:08:50.005310 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 15:08:50.005630 10476 heartbeater.cc:347] Connected to a master server at 127.30.8.62:36293
I20260811 15:08:50.005734 10476 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:50.005977 10476 heartbeater.cc:511] Master 127.30.8.62:36293 requested a full tablet report, sending...
I20260811 15:08:50.006728 8461 ts_manager.cc:194] Re-registered known tserver with Master: 0f34c101cebf4318a7484b40cdfc072a (127.30.8.3:45381)
I20260811 15:08:50.007449 8461 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:45097
I20260811 15:08:50.007949 10495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.992505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41458 (local address 127.30.8.62:36293)
0811 15:08:49.993288 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.993293 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.993426 (+ 133us) server_negotiation.cc:407] Connection header received
0811 15:08:49.993541 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.993544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.993605 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.993691 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.994756 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.995683 (+ 927us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.996924 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.997391 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.997491 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.999545 (+ 2054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:49.999549 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:49.999551 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:49.999577 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:50.001755 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:50.004092 (+ 2337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:50.004096 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:50.004098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:50.004190 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:50.005230 (+ 1040us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:50.005234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:50.005236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:50.005455 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:50.005674 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:50.007547 (+ 1873us) server_negotiation.cc:299] Negotiation successful
0811 15:08:50.007685 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":249,"thread_start_us":100,"threads_started":1}
I20260811 15:08:50.009931 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.982585 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:44293 (local address 127.30.8.60:33179)
0811 15:08:49.982798 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.982802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.984921 (+ 2119us) server_negotiation.cc:407] Connection header received
0811 15:08:49.985106 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.985109 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.985162 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.985217 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.986716 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.987573 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.989014 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.989373 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.989475 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:50.006074 (+ 16599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:50.006077 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:50.006080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:50.006106 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:50.007915 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:50.008421 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:50.008424 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:50.008426 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:50.008515 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:50.008842 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:50.008846 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:50.008847 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:50.009058 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:50.009131 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:50.009655 (+ 524us) server_negotiation.cc:299] Negotiation successful
0811 15:08:50.009794 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260811 15:08:50.010540 10482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.982858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:33179 (local address 127.30.8.3:44293)
0811 15:08:49.984829 (+ 1971us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:49.984843 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:49.984986 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:49.985376 (+ 390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:49.985379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:49.985688 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:49.986537 (+ 849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.986560 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.987709 (+ 1149us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.987713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.988887 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:49.988897 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.989636 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.989640 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:49.989856 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:49.990712 (+ 856us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:49.990727 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:50.005925 (+ 15198us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:50.008044 (+ 2119us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:50.008048 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:50.008051 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:50.008301 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:50.008631 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:50.008633 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:50.008635 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:50.008722 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:50.009205 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:50.009208 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:50.009451 (+ 243us) client_negotiation.cc:769] Sending connection context
0811 15:08:50.010078 (+ 627us) client_negotiation.cc:240] Negotiation successful
0811 15:08:50.010351 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1902,"mutex_wait_us":258,"spinlock_wait_cycles":55680,"thread_start_us":135,"threads_started":1}
I20260811 15:08:50.011253 10475 heartbeater.cc:347] Connected to a master server at 127.30.8.60:33179
I20260811 15:08:50.011363 10475 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:50.011615 10475 heartbeater.cc:511] Master 127.30.8.60:33179 requested a full tablet report, sending...
I20260811 15:08:50.012377 9773 ts_manager.cc:194] Registered new tserver with Master: 0f34c101cebf4318a7484b40cdfc072a (127.30.8.3:45381)
I20260811 15:08:50.013197 10497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:49.992965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52406 (local address 127.30.8.61:40131)
0811 15:08:49.993956 (+ 991us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:49.993959 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:49.994334 (+ 375us) server_negotiation.cc:407] Connection header received
0811 15:08:49.994376 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:49.994379 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:49.994421 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:49.994497 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:49.995603 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.996359 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.997607 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:49.998329 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:49.998419 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:50.003329 (+ 4910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:50.003333 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:50.003335 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:50.003369 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:50.008469 (+ 5100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:50.011135 (+ 2666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:50.011139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:50.011140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:50.011265 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:50.011616 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:50.011620 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:50.011622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:50.011936 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:50.012078 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:50.012854 (+ 776us) server_negotiation.cc:299] Negotiation successful
0811 15:08:50.013011 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":922,"thread_start_us":114,"threads_started":1}
I20260811 15:08:50.010746 8461 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41458:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:08:50.014880 8461 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:08:50.017586 9719 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: NON_VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: true } } }
I20260811 15:08:50.017751 9719 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:50.022086 10501 raft_consensus.cc:1050] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f: attempting to promote NON_VOTER 660b3ae52063414f8fbd6024f6b8296a to VOTER
I20260811 15:08:50.022516 10501 consensus_queue.cc:237] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false } }
I20260811 15:08:50.023883 9486 raft_consensus.cc:1261] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 1 FOLLOWER]: Refusing update from remote peer c7378c939fdc4fd38b8eaab16d60d54f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 15:08:50.024353 9790 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 1 LEARNER]: Refusing update from remote peer c7378c939fdc4fd38b8eaab16d60d54f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 15:08:50.024338 10504 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 15:08:50.024884 10504 consensus_queue.cc:1035] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [LEADER]: Connected to new peer: Peer: permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 15:08:50.026392 9719 raft_consensus.cc:2932] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 660b3ae52063414f8fbd6024f6b8296a (127.30.8.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false } } }
I20260811 15:08:50.026798 9486 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 660b3ae52063414f8fbd6024f6b8296a (127.30.8.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false } } }
I20260811 15:08:50.027339 9790 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 660b3ae52063414f8fbd6024f6b8296a (127.30.8.60) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false } } }
I20260811 15:08:50.028151 9719 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false } } }
I20260811 15:08:50.028249 9719 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:50.028345 9719 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false } } }
I20260811 15:08:50.028399 9719 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:50.028445 10502 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false } } }
I20260811 15:08:50.028553 10502 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:50.029690 10519 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false } } }
I20260811 15:08:50.029803 10519 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:08:50.031289 9833 sys_catalog.cc:455] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false } } }
I20260811 15:08:50.031417 9833 sys_catalog.cc:458] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:08:50.036652 9386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:50.030449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:41249 (local address 127.0.0.1:51416)
0811 15:08:50.030620 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:50.030635 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:50.030722 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:50.032819 (+ 2097us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:50.032822 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:50.032843 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:50.032957 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:50.032962 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.033999 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.034002 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:50.035024 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:50.035035 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.036222 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.036226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:50.036320 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:50.036347 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 15:08:50.036426 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 15:08:50.036499 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 15:08:50.036733 10520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:50.030635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51416 (local address 127.30.8.2:41249)
0811 15:08:50.032451 (+ 1816us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:50.032461 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:50.032480 (+ 19us) server_negotiation.cc:407] Connection header received
0811 15:08:50.032543 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:50.032547 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:50.032702 (+ 155us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:50.032832 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:50.033084 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.033865 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.035207 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.036070 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.036193 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:50.036362 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:50.036462 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 15:08:50.036574 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1717,"thread_start_us":150,"threads_started":1}
I20260811 15:08:50.037024 10523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:50.030684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:37731 (local address 127.0.0.1:34298)
0811 15:08:50.031338 (+ 654us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:50.031352 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:50.031443 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:50.033032 (+ 1589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:50.033036 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:50.033056 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:50.033150 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:50.033156 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.034458 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.034461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:50.035585 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:50.035606 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.036653 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.036657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:50.036744 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:50.036770 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 15:08:50.036820 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 15:08:50.036897 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":111,"threads_started":1}
I20260811 15:08:50.037614 10531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:50.031394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34298 (local address 127.30.8.1:37731)
0811 15:08:50.031726 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:50.031735 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:50.031764 (+ 29us) server_negotiation.cc:407] Connection header received
0811 15:08:50.031850 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:50.031856 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:50.032123 (+ 267us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:50.033065 (+ 942us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:50.033288 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.034332 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.035746 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.036501 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.037192 (+ 691us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:50.037334 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:50.037383 (+ 49us) server_negotiation.cc:299] Negotiation successful
0811 15:08:50.037471 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":87,"threads_started":1}
I20260811 15:08:50.038115 10526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:50.030967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:45381 (local address 127.0.0.1:44604)
0811 15:08:50.033317 (+ 2350us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:50.033331 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:50.033413 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:50.034314 (+ 901us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:50.034317 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:50.034337 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:50.034443 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:50.034450 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.035784 (+ 1334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.035788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:50.036785 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:50.036796 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.037801 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.037805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:50.037894 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:50.037917 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 15:08:50.037957 (+ 40us) client_negotiation.cc:240] Negotiation successful
0811 15:08:50.038014 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2285,"mutex_wait_us":97,"thread_start_us":95,"threads_started":1}
I20260811 15:08:50.038782 10529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:50.031101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44604 (local address 127.30.8.3:45381)
0811 15:08:50.033687 (+ 2586us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:50.033698 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:50.033720 (+ 22us) server_negotiation.cc:407] Connection header received
0811 15:08:50.033795 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:50.033799 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:50.033974 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:50.034576 (+ 602us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:50.034694 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.035652 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.036945 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.037639 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.038401 (+ 762us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:50.038544 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:50.038589 (+ 45us) server_negotiation.cc:299] Negotiation successful
0811 15:08:50.038665 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2476,"thread_start_us":89,"threads_started":1}
I20260811 15:08:50.040405 9952 tablet_service.cc:1467] Processing CreateTablet for tablet 0c03bdc7a74c45d2a1274debb658d969 (DEFAULT_TABLE table=test-table [id=e7679b6ecc5348be9c03341f2e7f915f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:50.040418 10153 tablet_service.cc:1467] Processing CreateTablet for tablet 0c03bdc7a74c45d2a1274debb658d969 (DEFAULT_TABLE table=test-table [id=e7679b6ecc5348be9c03341f2e7f915f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:50.041637 9952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c03bdc7a74c45d2a1274debb658d969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:50.041637 10153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c03bdc7a74c45d2a1274debb658d969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:50.041385 10371 tablet_service.cc:1467] Processing CreateTablet for tablet 0c03bdc7a74c45d2a1274debb658d969 (DEFAULT_TABLE table=test-table [id=e7679b6ecc5348be9c03341f2e7f915f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:50.042212 10371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c03bdc7a74c45d2a1274debb658d969. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:50.049683 10533 tablet_bootstrap.cc:492] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a: Bootstrap starting.
I20260811 15:08:50.050732 10539 tablet_bootstrap.cc:492] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc: Bootstrap starting.
I20260811 15:08:50.052906 10539 tablet_bootstrap.cc:654] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:50.052904 10533 tablet_bootstrap.cc:654] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:50.053692 10539 log.cc:824] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:50.053692 10533 log.cc:824] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:50.054481 10538 tablet_bootstrap.cc:492] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd: Bootstrap starting.
I20260811 15:08:50.055930 10533 tablet_bootstrap.cc:492] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a: No bootstrap required, opened a new log
I20260811 15:08:50.056162 10533 ts_tablet_manager.cc:1397] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 15:08:50.056373 10539 tablet_bootstrap.cc:492] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc: No bootstrap required, opened a new log
I20260811 15:08:50.056557 10539 ts_tablet_manager.cc:1397] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 15:08:50.057080 10538 tablet_bootstrap.cc:654] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:50.058140 10538 log.cc:824] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:50.060472 10539 raft_consensus.cc:348] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 } } peers { permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 } } peers { permanent_uuid: "0f34c101cebf4318a7484b40cdfc072a" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45381 } }
I20260811 15:08:50.060449 10533 raft_consensus.cc:348] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 } } peers { permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 } } peers { permanent_uuid: "0f34c101cebf4318a7484b40cdfc072a" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45381 } }
I20260811 15:08:50.060703 10539 raft_consensus.cc:374] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:50.060704 10533 raft_consensus.cc:374] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:50.060747 10539 raft_consensus.cc:729] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fea2d50e2519476984b8fec3e8d172bc, State: Initialized, Role: FOLLOWER
I20260811 15:08:50.060766 10533 raft_consensus.cc:729] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f34c101cebf4318a7484b40cdfc072a, State: Initialized, Role: FOLLOWER
I20260811 15:08:50.061221 10539 consensus_queue.cc:260] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 } } peers { permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 } } peers { permanent_uuid: "0f34c101cebf4318a7484b40cdfc072a" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45381 } }
I20260811 15:08:50.061221 10533 consensus_queue.cc:260] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 } } peers { permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 } } peers { permanent_uuid: "0f34c101cebf4318a7484b40cdfc072a" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45381 } }
I20260811 15:08:50.061947 10476 heartbeater.cc:503] Master 127.30.8.62:36293 was elected leader, sending a full tablet report...
I20260811 15:08:50.062328 10533 ts_tablet_manager.cc:1428] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 15:08:50.062330 10539 ts_tablet_manager.cc:1428] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 15:08:50.063114 10254 heartbeater.cc:503] Master 127.30.8.62:36293 was elected leader, sending a full tablet report...
I20260811 15:08:50.069762 10538 tablet_bootstrap.cc:492] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd: No bootstrap required, opened a new log
I20260811 15:08:50.070019 10538 ts_tablet_manager.cc:1397] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd: Time spent bootstrapping tablet: real 0.016s user 0.004s sys 0.000s
I20260811 15:08:50.073346 10538 raft_consensus.cc:348] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 } } peers { permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 } } peers { permanent_uuid: "0f34c101cebf4318a7484b40cdfc072a" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45381 } }
I20260811 15:08:50.073630 10538 raft_consensus.cc:374] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:50.073694 10538 raft_consensus.cc:729] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b439d6b462a49188ab751501b266fdd, State: Initialized, Role: FOLLOWER
I20260811 15:08:50.074187 10538 consensus_queue.cc:260] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 } } peers { permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 } } peers { permanent_uuid: "0f34c101cebf4318a7484b40cdfc072a" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45381 } }
I20260811 15:08:50.074977 10045 heartbeater.cc:503] Master 127.30.8.62:36293 was elected leader, sending a full tablet report...
I20260811 15:08:50.075344 10538 ts_tablet_manager.cc:1428] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260811 15:08:50.164433 10546 raft_consensus.cc:482] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:08:50.164837 10546 raft_consensus.cc:504] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 } } peers { permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 } } peers { permanent_uuid: "0f34c101cebf4318a7484b40cdfc072a" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45381 } }
I20260811 15:08:50.166447 10546 leader_election.cc:288] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fea2d50e2519476984b8fec3e8d172bc (127.30.8.2:41249), 8b439d6b462a49188ab751501b266fdd (127.30.8.1:37731)
I20260811 15:08:50.171067 10520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:50.166993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:58759 (local address 127.30.8.2:41249)
0811 15:08:50.167167 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:50.167173 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:50.167278 (+ 105us) server_negotiation.cc:407] Connection header received
0811 15:08:50.167486 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:50.167489 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:50.167547 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:50.167615 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:50.167997 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.168595 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.169900 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.170351 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.170457 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:50.170544 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:50.170786 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 15:08:50.170880 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 15:08:50.171057 10531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:50.167323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:50085 (local address 127.30.8.1:37731)
0811 15:08:50.167528 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:50.167533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:50.167547 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:08:50.167595 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:50.167597 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:50.167640 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:50.167706 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:50.168073 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.168594 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.169928 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.170337 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.170436 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:50.170517 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:50.170771 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 15:08:50.170862 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 15:08:50.171090 10484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:50.166813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:41249 (local address 127.30.8.3:58759)
0811 15:08:50.167162 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:50.167180 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:50.167329 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:50.167673 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:50.167676 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:50.167699 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:50.167864 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:50.167871 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.168732 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.168735 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:50.169738 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:50.169757 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.170495 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.170498 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:50.170644 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:50.170666 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 15:08:50.170740 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 15:08:50.170865 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":62}
I20260811 15:08:50.171087 10482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:50.166993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:37731 (local address 127.30.8.3:50085)
0811 15:08:50.167199 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:50.167217 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:50.167366 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:50.167762 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:50.167765 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:50.167789 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:50.167942 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:50.167949 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.168730 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.168733 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:50.169773 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:50.169785 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.170483 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.170487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:50.170625 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:50.170651 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 15:08:50.170722 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 15:08:50.170846 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":83,"spinlock_wait_cycles":896}
I20260811 15:08:50.172166 10181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c03bdc7a74c45d2a1274debb658d969" candidate_uuid: "0f34c101cebf4318a7484b40cdfc072a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea2d50e2519476984b8fec3e8d172bc" is_pre_election: true
I20260811 15:08:50.172519 9978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c03bdc7a74c45d2a1274debb658d969" candidate_uuid: "0f34c101cebf4318a7484b40cdfc072a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b439d6b462a49188ab751501b266fdd" is_pre_election: true
I20260811 15:08:50.172544 10181 raft_consensus.cc:2454] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34c101cebf4318a7484b40cdfc072a in term 0.
I20260811 15:08:50.172775 9978 raft_consensus.cc:2454] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34c101cebf4318a7484b40cdfc072a in term 0.
I20260811 15:08:50.173138 10294 leader_election.cc:302] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34c101cebf4318a7484b40cdfc072a, fea2d50e2519476984b8fec3e8d172bc; no voters:
I20260811 15:08:50.173480 10546 raft_consensus.cc:2781] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:08:50.173569 10546 raft_consensus.cc:482] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:08:50.173601 10546 raft_consensus.cc:3037] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:50.174728 10546 raft_consensus.cc:504] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 } } peers { permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 } } peers { permanent_uuid: "0f34c101cebf4318a7484b40cdfc072a" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45381 } }
I20260811 15:08:50.175163 10546 leader_election.cc:288] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [CANDIDATE]: Term 1 election: Requested vote from peers fea2d50e2519476984b8fec3e8d172bc (127.30.8.2:41249), 8b439d6b462a49188ab751501b266fdd (127.30.8.1:37731)
I20260811 15:08:50.175593 9978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c03bdc7a74c45d2a1274debb658d969" candidate_uuid: "0f34c101cebf4318a7484b40cdfc072a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b439d6b462a49188ab751501b266fdd"
I20260811 15:08:50.175602 10181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c03bdc7a74c45d2a1274debb658d969" candidate_uuid: "0f34c101cebf4318a7484b40cdfc072a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea2d50e2519476984b8fec3e8d172bc"
I20260811 15:08:50.175740 10181 raft_consensus.cc:3037] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:50.175740 9978 raft_consensus.cc:3037] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:50.177119 9978 raft_consensus.cc:2454] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34c101cebf4318a7484b40cdfc072a in term 1.
I20260811 15:08:50.177114 10181 raft_consensus.cc:2454] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34c101cebf4318a7484b40cdfc072a in term 1.
I20260811 15:08:50.177443 10294 leader_election.cc:302] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0f34c101cebf4318a7484b40cdfc072a, 8b439d6b462a49188ab751501b266fdd, fea2d50e2519476984b8fec3e8d172bc; no voters:
I20260811 15:08:50.177623 10546 raft_consensus.cc:2781] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:50.177857 10546 raft_consensus.cc:686] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [term 1 LEADER]: Becoming Leader. State: Replica: 0f34c101cebf4318a7484b40cdfc072a, State: Running, Role: LEADER
I20260811 15:08:50.178282 10546 consensus_queue.cc:237] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 } } peers { permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 } } peers { permanent_uuid: "0f34c101cebf4318a7484b40cdfc072a" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45381 } }
I20260811 15:08:50.182499 8459 catalog_manager.cc:5495] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f34c101cebf4318a7484b40cdfc072a (127.30.8.3). New cstate: current_term: 1 leader_uuid: "0f34c101cebf4318a7484b40cdfc072a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f34c101cebf4318a7484b40cdfc072a" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45381 } health_report { overall_health: HEALTHY } } }
I20260811 15:08:50.228802 10529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:50.224804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44610 (local address 127.30.8.3:45381)
0811 15:08:50.224971 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:50.224975 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:50.224991 (+ 16us) server_negotiation.cc:407] Connection header received
0811 15:08:50.225110 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:50.225113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:50.225191 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:50.225311 (+ 120us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:50.225566 (+ 255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.226149 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.227039 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.227283 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.227389 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:50.227854 (+ 465us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:50.228379 (+ 525us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:50.228491 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:50.228588 (+ 97us) server_negotiation.cc:299] Negotiation successful
0811 15:08:50.228663 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 15:08:50.230988 10479 tablet.cc:2369] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:08:50.235558 10181 raft_consensus.cc:1261] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc [term 1 FOLLOWER]: Refusing update from remote peer 0f34c101cebf4318a7484b40cdfc072a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:50.235558 9978 raft_consensus.cc:1261] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd [term 1 FOLLOWER]: Refusing update from remote peer 0f34c101cebf4318a7484b40cdfc072a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:50.236454 10549 consensus_queue.cc:1035] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:50.236660 10546 consensus_queue.cc:1035] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 15:08:50.238993 10047 tablet.cc:2369] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 15:08:50.241997 10257 tablet.cc:2369] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:08:50.247644 10557 mvcc.cc:205] Tried to move back new op lower bound from 7317343970236366848 to 7317343970013769728. Current Snapshot: MvccSnapshot[applied={T|T < 7317343970236366848}]
I20260811 15:08:50.251122 10556 mvcc.cc:205] Tried to move back new op lower bound from 7317343970236366848 to 7317343970013769728. Current Snapshot: MvccSnapshot[applied={T|T < 7317343970236366848}]
I20260811 15:08:50.260803 8459 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:50.260998 8459 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41458:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:50.265872 8459 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c7378c939fdc4fd38b8eaab16d60d54f: Sending DeleteTablet for 3 replicas of tablet 0c03bdc7a74c45d2a1274debb658d969
I20260811 15:08:50.266652 10153 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c03bdc7a74c45d2a1274debb658d969 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:50 UTC) from {username='oryx', principal='oryx/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:51416
I20260811 15:08:50.266983 9952 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c03bdc7a74c45d2a1274debb658d969 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:50 UTC) from {username='oryx', principal='oryx/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:34298
I20260811 15:08:50.267201 10573 tablet_replica.cc:330] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc: stopping tablet replica
I20260811 15:08:50.267356 10371 tablet_service.cc:1514] Processing DeleteTablet for tablet 0c03bdc7a74c45d2a1274debb658d969 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:50 UTC) from {username='oryx', principal='oryx/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:44604
I20260811 15:08:50.267477 10574 tablet_replica.cc:330] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd: stopping tablet replica
I20260811 15:08:50.267585 10573 raft_consensus.cc:2229] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:50.267774 10574 raft_consensus.cc:2229] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:50.267985 10573 raft_consensus.cc:2258] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:50.268061 10574 raft_consensus.cc:2258] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:50.268373 10575 tablet_replica.cc:330] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a: stopping tablet replica
I20260811 15:08:50.268658 10575 raft_consensus.cc:2229] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:08:50.269950 10575 raft_consensus.cc:2258] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:50.270851 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 8409
I20260811 15:08:50.275533 10573 ts_tablet_manager.cc:1905] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:50.278446 10574 ts_tablet_manager.cc:1905] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:50.283641 10574 ts_tablet_manager.cc:1918] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:08:50.283771 10574 log.cc:1196] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-0/wal/wals/0c03bdc7a74c45d2a1274debb658d969
I20260811 15:08:50.284140 10574 ts_tablet_manager.cc:1939] T 0c03bdc7a74c45d2a1274debb658d969 P 8b439d6b462a49188ab751501b266fdd: Deleting consensus metadata
I20260811 15:08:50.285003 10575 ts_tablet_manager.cc:1905] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:50.287228 10573 ts_tablet_manager.cc:1918] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:08:50.287365 10573 log.cc:1196] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/0c03bdc7a74c45d2a1274debb658d969
I20260811 15:08:50.287722 10573 ts_tablet_manager.cc:1939] T 0c03bdc7a74c45d2a1274debb658d969 P fea2d50e2519476984b8fec3e8d172bc: Deleting consensus metadata
I20260811 15:08:50.288267 10575 ts_tablet_manager.cc:1918] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:08:50.288375 10575 log.cc:1196] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/0c03bdc7a74c45d2a1274debb658d969
I20260811 15:08:50.288702 10575 ts_tablet_manager.cc:1939] T 0c03bdc7a74c45d2a1274debb658d969 P 0f34c101cebf4318a7484b40cdfc072a: Deleting consensus metadata
W20260811 15:08:50.295588 10295 connection.cc:535] server connection from 127.0.0.1:44604 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 15:08:50.295614 9873 connection.cc:535] server connection from 127.0.0.1:34298 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 15:08:50.296907 10088 connection.cc:535] server connection from 127.0.0.1:51416 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 15:08:50.299188 10497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:50.270135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52418 (local address 127.30.8.61:40131)
0811 15:08:50.270394 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:50.270399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:50.270413 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:08:50.270555 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:50.270559 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:50.270624 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:50.270679 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:50.285554 (+ 14875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.286584 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.288020 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.288353 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.288432 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:50.294533 (+ 6101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:50.294539 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:50.294542 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:50.294587 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:50.296670 (+ 2083us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:50.297587 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:50.297592 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:50.297594 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:50.297699 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:50.298081 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:50.298084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:50.298086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:50.298382 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:50.298529 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:50.298847 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 15:08:50.299020 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181}
I20260811 15:08:50.309746 10048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:50.270265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50580 (local address 127.30.8.60:33179)
0811 15:08:50.270507 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:50.270511 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:50.299814 (+ 29303us) server_negotiation.cc:407] Connection header received
0811 15:08:50.300005 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:50.300008 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:50.300071 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:50.300122 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:50.301232 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.302065 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.303265 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.303608 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.303680 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:50.305436 (+ 1756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:50.305441 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:50.305444 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:50.305470 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:50.307745 (+ 2275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:50.308201 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:50.308206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:50.308208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:50.308301 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:50.308630 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:50.308634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:50.308636 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:50.308896 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:50.309013 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:50.309411 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 15:08:50.309559 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166}
I20260811 15:08:50.311197 10261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:50.300663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50582 (local address 127.30.8.60:33179)
0811 15:08:50.300880 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:50.300883 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:50.301549 (+ 666us) server_negotiation.cc:407] Connection header received
0811 15:08:50.301724 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:50.301727 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:50.301776 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:50.301829 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:50.302653 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.303451 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.304659 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.305003 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.305066 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:50.306969 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:50.306973 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:50.306976 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:50.307002 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:50.308869 (+ 1867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:50.309334 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:50.309338 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:50.309340 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:50.309421 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:50.309718 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:50.309722 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:50.309723 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:50.309921 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:50.310014 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:50.310835 (+ 821us) server_negotiation.cc:299] Negotiation successful
0811 15:08:50.311044 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151}
I20260811 15:08:50.311344 10497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:50.300544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52424 (local address 127.30.8.61:40131)
0811 15:08:50.300792 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:50.300796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:50.301290 (+ 494us) server_negotiation.cc:407] Connection header received
0811 15:08:50.301473 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:50.301477 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:50.301526 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:50.301575 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:50.302536 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.303364 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.304646 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:50.304988 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:50.305062 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:50.307779 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:50.307783 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:50.307786 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:50.307817 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:50.309371 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:50.309829 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:50.309834 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:50.309836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:50.309927 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:50.310269 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:50.310273 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:50.310274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:50.310538 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:50.310649 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:50.311074 (+ 425us) server_negotiation.cc:299] Negotiation successful
0811 15:08:50.311203 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260811 15:08:51.250353 10589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:51.249787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.1:54829)
0811 15:08:51.250129 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:51.250237 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":87,"threads_started":1}
I20260811 15:08:51.250676 10590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:51.249998 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.2:56879)
0811 15:08:51.250254 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:51.250348 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":78,"threads_started":1}
W20260811 15:08:51.250696 10045 heartbeater.cc:644] Failed to heartbeat to 127.30.8.62:36293 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
W20260811 15:08:51.251042 10254 heartbeater.cc:644] Failed to heartbeat to 127.30.8.62:36293 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
I20260811 15:08:51.252724 10591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:51.252273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.3:35271)
0811 15:08:51.252521 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:51.252622 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":77,"threads_started":1}
W20260811 15:08:51.252971 10476 heartbeater.cc:644] Failed to heartbeat to 127.30.8.62:36293 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
I20260811 15:08:51.766103 10592 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7378c939fdc4fd38b8eaab16d60d54f)
I20260811 15:08:51.766424 10593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 1 FOLLOWER]: Starting pre-election (detected failure of leader c7378c939fdc4fd38b8eaab16d60d54f)
I20260811 15:08:51.766412 10592 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false } }
I20260811 15:08:51.766621 10593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false } }
I20260811 15:08:51.767588 10593 leader_election.cc:288] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7378c939fdc4fd38b8eaab16d60d54f (127.30.8.62:36293), dd9736da73324408b58ba279deb3f85e (127.30.8.61:40131)
I20260811 15:08:51.768023 10592 leader_election.cc:288] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c7378c939fdc4fd38b8eaab16d60d54f (127.30.8.62:36293), 660b3ae52063414f8fbd6024f6b8296a (127.30.8.60:33179)
I20260811 15:08:51.768451 10596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:51.767808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.0.0.1:41926)
0811 15:08:51.768255 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:51.768340 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":71,"threads_started":1}
I20260811 15:08:51.768630 10601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:51.768219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.0.0.1:41932)
0811 15:08:51.768466 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:51.768536 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":76,"threads_started":1}
W20260811 15:08:51.768630 9753 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
W20260811 15:08:51.768811 9444 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
I20260811 15:08:51.769646 10596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:51.769393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.0.0.1:41948)
0811 15:08:51.769530 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:51.769577 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260811 15:08:51.769802 9753 leader_election.cc:334] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c7378c939fdc4fd38b8eaab16d60d54f (127.30.8.62:36293): Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
I20260811 15:08:51.770169 10601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:51.769951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.0.0.1:41956)
0811 15:08:51.770049 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:51.770102 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 15:08:51.770277 9444 leader_election.cc:334] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer c7378c939fdc4fd38b8eaab16d60d54f (127.30.8.62:36293): Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
I20260811 15:08:51.772789 10598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:51.768030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50596 (local address 127.30.8.60:33179)
0811 15:08:51.768394 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:51.768398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:51.768413 (+ 15us) server_negotiation.cc:407] Connection header received
0811 15:08:51.768489 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:51.768585 (+ 96us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:51.768644 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:51.768857 (+ 213us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:51.769307 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:51.769998 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:51.771579 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:51.772210 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:51.772325 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:51.772390 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:51.772556 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 15:08:51.772632 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":51,"threads_started":1}
I20260811 15:08:51.772784 10600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:51.768025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52436 (local address 127.30.8.61:40131)
0811 15:08:51.768288 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:51.768292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:51.768764 (+ 472us) server_negotiation.cc:407] Connection header received
0811 15:08:51.768811 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:51.768814 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:51.768861 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:51.768938 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:51.769329 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:51.770052 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:51.771579 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:51.772214 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:51.772325 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:51.772391 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:51.772556 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 15:08:51.772632 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":53,"threads_started":1}
I20260811 15:08:51.772794 10594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:51.767827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:40131 (local address 127.0.0.1:52436)
0811 15:08:51.768574 (+ 747us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:51.768587 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:51.768679 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:51.769067 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:51.769070 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:51.769088 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:51.769200 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:51.769208 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:51.770147 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:51.770150 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:51.771467 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:51.771476 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:51.772330 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:51.772334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:51.772431 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:51.772457 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 15:08:51.772534 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 15:08:51.772610 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":686,"thread_start_us":67,"threads_started":1}
I20260811 15:08:51.772799 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:51.767808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:33179 (local address 127.0.0.1:50596)
0811 15:08:51.768259 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:51.768276 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:51.768384 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:51.768988 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:51.768991 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:51.769012 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:51.769124 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:51.769132 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:51.770118 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:51.770122 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:51.771447 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:51.771459 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:51.772330 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:51.772334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:51.772431 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:51.772457 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 15:08:51.772534 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 15:08:51.772610 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":85,"threads_started":1}
I20260811 15:08:51.773674 9790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dd9736da73324408b58ba279deb3f85e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "660b3ae52063414f8fbd6024f6b8296a" is_pre_election: true
I20260811 15:08:51.773895 9790 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd9736da73324408b58ba279deb3f85e in term 1.
I20260811 15:08:51.773777 9486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "660b3ae52063414f8fbd6024f6b8296a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "dd9736da73324408b58ba279deb3f85e" is_pre_election: true
I20260811 15:08:51.774197 9486 raft_consensus.cc:2454] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 660b3ae52063414f8fbd6024f6b8296a in term 1.
I20260811 15:08:51.774458 9450 leader_election.cc:302] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 660b3ae52063414f8fbd6024f6b8296a, dd9736da73324408b58ba279deb3f85e; no voters: c7378c939fdc4fd38b8eaab16d60d54f
I20260811 15:08:51.774720 10592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 15:08:51.774731 9755 leader_election.cc:302] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 660b3ae52063414f8fbd6024f6b8296a, dd9736da73324408b58ba279deb3f85e; no voters: c7378c939fdc4fd38b8eaab16d60d54f
I20260811 15:08:51.774783 10592 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 1 FOLLOWER]: Starting leader election (detected failure of leader c7378c939fdc4fd38b8eaab16d60d54f)
I20260811 15:08:51.774823 10592 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 1 FOLLOWER]: Advancing to term 2
I20260811 15:08:51.774987 10593 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 15:08:51.775048 10593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 1 FOLLOWER]: Starting leader election (detected failure of leader c7378c939fdc4fd38b8eaab16d60d54f)
I20260811 15:08:51.775094 10593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 1 FOLLOWER]: Advancing to term 2
I20260811 15:08:51.776211 10593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false } }
I20260811 15:08:51.776218 10592 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false } }
I20260811 15:08:51.776646 10592 leader_election.cc:288] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [CANDIDATE]: Term 2 election: Requested vote from peers c7378c939fdc4fd38b8eaab16d60d54f (127.30.8.62:36293), 660b3ae52063414f8fbd6024f6b8296a (127.30.8.60:33179)
I20260811 15:08:51.776790 10593 leader_election.cc:288] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [CANDIDATE]: Term 2 election: Requested vote from peers c7378c939fdc4fd38b8eaab16d60d54f (127.30.8.62:36293), dd9736da73324408b58ba279deb3f85e (127.30.8.61:40131)
I20260811 15:08:51.777052 10594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:51.776786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.0.0.1:41960)
0811 15:08:51.776938 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:51.776980 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260811 15:08:51.777073 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:51.776786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.0.0.1:41958)
0811 15:08:51.776920 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:51.776979 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 15:08:51.777166 9790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dd9736da73324408b58ba279deb3f85e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "660b3ae52063414f8fbd6024f6b8296a"
I20260811 15:08:51.777248 9486 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "660b3ae52063414f8fbd6024f6b8296a" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "dd9736da73324408b58ba279deb3f85e"
I20260811 15:08:51.777345 9790 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate dd9736da73324408b58ba279deb3f85e in current term 2: Already voted for candidate 660b3ae52063414f8fbd6024f6b8296a in this term.
I20260811 15:08:51.777402 9486 raft_consensus.cc:2379] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 660b3ae52063414f8fbd6024f6b8296a in current term 2: Already voted for candidate dd9736da73324408b58ba279deb3f85e in this term.
I20260811 15:08:51.777735 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:51.777527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.0.0.1:41988)
0811 15:08:51.777613 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:51.777664 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260811 15:08:51.777817 9444 leader_election.cc:334] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c7378c939fdc4fd38b8eaab16d60d54f (127.30.8.62:36293): Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
I20260811 15:08:51.777873 9444 leader_election.cc:302] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: dd9736da73324408b58ba279deb3f85e; no voters: 660b3ae52063414f8fbd6024f6b8296a, c7378c939fdc4fd38b8eaab16d60d54f
I20260811 15:08:51.778046 10592 raft_consensus.cc:2726] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 15:08:51.778266 10594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:51.778044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.0.0.1:41990)
0811 15:08:51.778151 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:51.778211 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 15:08:51.778368 9753 leader_election.cc:334] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer c7378c939fdc4fd38b8eaab16d60d54f (127.30.8.62:36293): Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
I20260811 15:08:51.778465 9753 leader_election.cc:302] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 660b3ae52063414f8fbd6024f6b8296a; no voters: c7378c939fdc4fd38b8eaab16d60d54f, dd9736da73324408b58ba279deb3f85e
I20260811 15:08:51.778579 10593 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260811 15:08:52.251894 10608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:52.251427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.1:48509)
0811 15:08:52.251709 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:52.251778 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":85,"threads_started":1}
I20260811 15:08:52.252617 10609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:52.251643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.2:46171)
0811 15:08:52.252459 (+ 816us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:52.252525 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":92,"threads_started":1}
I20260811 15:08:52.254314 10610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:52.253578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.3:52891)
0811 15:08:52.254150 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:52.254220 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":78,"threads_started":1}
I20260811 15:08:53.253378 10611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.252851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.1:56893)
0811 15:08:53.253203 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.253269 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":134,"threads_started":1}
I20260811 15:08:53.255607 10612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.253703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.2:38479)
0811 15:08:53.255373 (+ 1670us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.255460 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1566,"thread_start_us":126,"threads_started":1}
I20260811 15:08:53.255847 10613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.255253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.3:41351)
0811 15:08:53.255658 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.255729 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":81,"threads_started":1}
I20260811 15:08:53.757091 10614 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:08:53.757313 10614 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false } }
I20260811 15:08:53.757912 10614 leader_election.cc:288] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers c7378c939fdc4fd38b8eaab16d60d54f (127.30.8.62:36293), 660b3ae52063414f8fbd6024f6b8296a (127.30.8.60:33179)
I20260811 15:08:53.758643 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.758185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.0.0.1:42238)
0811 15:08:53.758443 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.758519 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":68,"threads_started":1}
I20260811 15:08:53.758633 9790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dd9736da73324408b58ba279deb3f85e" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "660b3ae52063414f8fbd6024f6b8296a" is_pre_election: true
I20260811 15:08:53.758846 9790 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dd9736da73324408b58ba279deb3f85e in term 2.
I20260811 15:08:53.759331 9450 leader_election.cc:302] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 660b3ae52063414f8fbd6024f6b8296a, dd9736da73324408b58ba279deb3f85e; no voters:
I20260811 15:08:53.759620 10614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260811 15:08:53.759697 10614 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:08:53.759768 10614 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 2 FOLLOWER]: Advancing to term 3
I20260811 15:08:53.759846 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.759552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.0.0.1:42242)
0811 15:08:53.759682 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.759770 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 15:08:53.760032 9444 leader_election.cc:334] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer c7378c939fdc4fd38b8eaab16d60d54f (127.30.8.62:36293): Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
I20260811 15:08:53.760953 10614 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false } }
I20260811 15:08:53.761363 10614 leader_election.cc:288] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [CANDIDATE]: Term 3 election: Requested vote from peers c7378c939fdc4fd38b8eaab16d60d54f (127.30.8.62:36293), 660b3ae52063414f8fbd6024f6b8296a (127.30.8.60:33179)
I20260811 15:08:53.761765 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.761546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.0.0.1:42246)
0811 15:08:53.761644 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.761696 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 15:08:53.761899 9790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "dd9736da73324408b58ba279deb3f85e" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "660b3ae52063414f8fbd6024f6b8296a"
I20260811 15:08:53.762048 9790 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 2 FOLLOWER]: Advancing to term 3
I20260811 15:08:53.762557 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.762319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.0.0.1:42262)
0811 15:08:53.762441 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.762496 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 15:08:53.762753 9444 leader_election.cc:334] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer c7378c939fdc4fd38b8eaab16d60d54f (127.30.8.62:36293): Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
I20260811 15:08:53.763316 9790 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dd9736da73324408b58ba279deb3f85e in term 3.
I20260811 15:08:53.763657 9450 leader_election.cc:302] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 660b3ae52063414f8fbd6024f6b8296a, dd9736da73324408b58ba279deb3f85e; no voters: c7378c939fdc4fd38b8eaab16d60d54f
I20260811 15:08:53.763844 10614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 3 FOLLOWER]: Leader election won for term 3
I20260811 15:08:53.764050 10614 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [term 3 LEADER]: Becoming Leader. State: Replica: dd9736da73324408b58ba279deb3f85e, State: Running, Role: LEADER
I20260811 15:08:53.764303 10614 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false } }
I20260811 15:08:53.765388 10618 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd9736da73324408b58ba279deb3f85e. Latest consensus state: current_term: 3 leader_uuid: "dd9736da73324408b58ba279deb3f85e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false } } }
I20260811 15:08:53.765516 10618 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:53.765826 10621 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:08:53.768121 10621 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:08:53.768543 10621 catalog_manager.cc:1259] Loaded cluster ID: 035c4a9125c04bba9b2b98a3ebcd918c
I20260811 15:08:53.768600 10621 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:08:53.769184 10621 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:08:53.769582 10621 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: Loaded TSK: 0
I20260811 15:08:53.770311 10621 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:08:53.790091 9469 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52424:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:08:53.793498 9469 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:08:53.796964 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.796633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.0.0.1:42274)
0811 15:08:53.796760 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.796847 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 15:08:53.797067 9790 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [term 3 FOLLOWER]: Refusing update from remote peer dd9736da73324408b58ba279deb3f85e: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260811 15:08:53.797605 10618 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [LEADER]: Connected to new peer: Peer: permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260811 15:08:53.797756 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.797504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.0.0.1:42286)
0811 15:08:53.797640 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.797696 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 15:08:53.798007 9444 consensus_peers.cc:487] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e -> Peer c7378c939fdc4fd38b8eaab16d60d54f (127.30.8.62:36293): Couldn't send request to peer c7378c939fdc4fd38b8eaab16d60d54f. Status: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 15:08:53.799340 10624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd9736da73324408b58ba279deb3f85e. Latest consensus state: current_term: 3 leader_uuid: "dd9736da73324408b58ba279deb3f85e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false } } }
I20260811 15:08:53.799470 10624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:08:53.800607 10614 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "dd9736da73324408b58ba279deb3f85e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false } } }
I20260811 15:08:53.800745 10614 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:53.801024 10624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "dd9736da73324408b58ba279deb3f85e" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "c7378c939fdc4fd38b8eaab16d60d54f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 36293 } } peers { permanent_uuid: "dd9736da73324408b58ba279deb3f85e" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 40131 } attrs { promote: false } } peers { permanent_uuid: "660b3ae52063414f8fbd6024f6b8296a" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 33179 } attrs { promote: false } } }
I20260811 15:08:53.801210 10625 mvcc.cc:205] Tried to move back new op lower bound from 7317343984827146240 to 7317343984701165568. Current Snapshot: MvccSnapshot[applied={T|T < 7317343984827146240}]
I20260811 15:08:53.801328 10624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 660b3ae52063414f8fbd6024f6b8296a [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:08:53.811542 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.806645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:45381 (local address 127.0.0.1:44624)
0811 15:08:53.807034 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.807050 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:53.807149 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:53.807699 (+ 550us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:53.807702 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:53.807747 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:53.807879 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:53.807886 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:53.808948 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:53.808951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:53.810178 (+ 1227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:53.810189 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:53.811098 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:53.811102 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:53.811191 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:53.811213 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 15:08:53.811302 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 15:08:53.811383 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"spinlock_wait_cycles":3840}
I20260811 15:08:53.811548 10629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.806774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44624 (local address 127.30.8.3:45381)
0811 15:08:53.807081 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:53.807086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:53.807100 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:08:53.807259 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:53.807262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:53.807315 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:53.807989 (+ 674us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:53.808101 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:53.808832 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:53.810320 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:53.810976 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:53.811077 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:53.811150 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:53.811327 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 15:08:53.811405 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":82,"threads_started":1}
I20260811 15:08:53.811862 10639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.807566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51430 (local address 127.30.8.2:41249)
0811 15:08:53.808080 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:53.808085 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:53.808098 (+ 13us) server_negotiation.cc:407] Connection header received
0811 15:08:53.808149 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:53.808152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:53.808198 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:53.808301 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:53.808555 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:53.809199 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:53.810518 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:53.811038 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:53.811143 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:53.811213 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:53.811596 (+ 383us) server_negotiation.cc:299] Negotiation successful
0811 15:08:53.811684 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":64,"threads_started":1}
I20260811 15:08:53.811967 10630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.807388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:37731 (local address 127.0.0.1:34314)
0811 15:08:53.807794 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.807808 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:53.807886 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:53.808456 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:53.808458 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:53.808472 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:53.808556 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:53.808562 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:53.809483 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:53.809487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:53.810638 (+ 1151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:53.810648 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:53.811606 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:53.811610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:53.811695 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:53.811720 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 15:08:53.811790 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 15:08:53.811854 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":4,"thread_start_us":98,"threads_started":1}
I20260811 15:08:53.812227 10637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.806942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:41249 (local address 127.0.0.1:51430)
0811 15:08:53.807629 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.807642 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:53.807744 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:53.808330 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:53.808334 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:53.808351 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:53.808450 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:53.808456 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:53.809316 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:53.809319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:53.810388 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:53.810398 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:53.811268 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:53.811272 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:53.811367 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:53.811387 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 15:08:53.811447 (+ 60us) client_negotiation.cc:240] Negotiation successful
0811 15:08:53.811505 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":623,"mutex_wait_us":365,"spinlock_wait_cycles":6912,"thread_start_us":134,"threads_started":1}
I20260811 15:08:53.812320 10638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.807530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34314 (local address 127.30.8.1:37731)
0811 15:08:53.808171 (+ 641us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:53.808174 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:53.808186 (+ 12us) server_negotiation.cc:407] Connection header received
0811 15:08:53.808234 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:53.808236 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:53.808274 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:53.808354 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:53.808670 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:53.809355 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:53.810768 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:53.811387 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:53.811484 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:53.811544 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:53.812138 (+ 594us) server_negotiation.cc:299] Negotiation successful
0811 15:08:53.812213 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":77,"threads_started":1}
I20260811 15:08:53.812582 10371 tablet_service.cc:1467] Processing CreateTablet for tablet f12bc7d3247649d4bf4eec6704b1a7c8 (DEFAULT_TABLE table=test-table [id=3457d3edcb444e598d0a2cd2093d571e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:53.812822 10153 tablet_service.cc:1467] Processing CreateTablet for tablet f12bc7d3247649d4bf4eec6704b1a7c8 (DEFAULT_TABLE table=test-table [id=3457d3edcb444e598d0a2cd2093d571e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:53.812927 10371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f12bc7d3247649d4bf4eec6704b1a7c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:53.813076 10153 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f12bc7d3247649d4bf4eec6704b1a7c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:53.815505 9952 tablet_service.cc:1467] Processing CreateTablet for tablet f12bc7d3247649d4bf4eec6704b1a7c8 (DEFAULT_TABLE table=test-table [id=3457d3edcb444e598d0a2cd2093d571e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:53.815724 10642 tablet_bootstrap.cc:492] T f12bc7d3247649d4bf4eec6704b1a7c8 P fea2d50e2519476984b8fec3e8d172bc: Bootstrap starting.
I20260811 15:08:53.815830 9952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f12bc7d3247649d4bf4eec6704b1a7c8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:53.816673 10642 tablet_bootstrap.cc:654] T f12bc7d3247649d4bf4eec6704b1a7c8 P fea2d50e2519476984b8fec3e8d172bc: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:53.817353 10640 tablet_bootstrap.cc:492] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a: Bootstrap starting.
I20260811 15:08:53.817881 10642 tablet_bootstrap.cc:492] T f12bc7d3247649d4bf4eec6704b1a7c8 P fea2d50e2519476984b8fec3e8d172bc: No bootstrap required, opened a new log
I20260811 15:08:53.817951 10642 ts_tablet_manager.cc:1397] T f12bc7d3247649d4bf4eec6704b1a7c8 P fea2d50e2519476984b8fec3e8d172bc: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 15:08:53.818272 10640 tablet_bootstrap.cc:654] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:53.818332 10642 raft_consensus.cc:348] T f12bc7d3247649d4bf4eec6704b1a7c8 P fea2d50e2519476984b8fec3e8d172bc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f34c101cebf4318a7484b40cdfc072a" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45381 } } peers { permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 } } peers { permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 } }
I20260811 15:08:53.818447 10642 raft_consensus.cc:374] T f12bc7d3247649d4bf4eec6704b1a7c8 P fea2d50e2519476984b8fec3e8d172bc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:53.818457 10647 tablet_bootstrap.cc:492] T f12bc7d3247649d4bf4eec6704b1a7c8 P 8b439d6b462a49188ab751501b266fdd: Bootstrap starting.
I20260811 15:08:53.818480 10642 raft_consensus.cc:729] T f12bc7d3247649d4bf4eec6704b1a7c8 P fea2d50e2519476984b8fec3e8d172bc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fea2d50e2519476984b8fec3e8d172bc, State: Initialized, Role: FOLLOWER
I20260811 15:08:53.818960 10642 consensus_queue.cc:260] T f12bc7d3247649d4bf4eec6704b1a7c8 P fea2d50e2519476984b8fec3e8d172bc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f34c101cebf4318a7484b40cdfc072a" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45381 } } peers { permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 } } peers { permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 } }
I20260811 15:08:53.819348 10642 ts_tablet_manager.cc:1428] T f12bc7d3247649d4bf4eec6704b1a7c8 P fea2d50e2519476984b8fec3e8d172bc: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:53.819407 10647 tablet_bootstrap.cc:654] T f12bc7d3247649d4bf4eec6704b1a7c8 P 8b439d6b462a49188ab751501b266fdd: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:53.820343 10654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.819894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.2:33521)
0811 15:08:53.820159 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.820225 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":100,"threads_started":1}
I20260811 15:08:53.820698 10647 tablet_bootstrap.cc:492] T f12bc7d3247649d4bf4eec6704b1a7c8 P 8b439d6b462a49188ab751501b266fdd: No bootstrap required, opened a new log
I20260811 15:08:53.820755 10640 tablet_bootstrap.cc:492] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a: No bootstrap required, opened a new log
I20260811 15:08:53.820784 10647 ts_tablet_manager.cc:1397] T f12bc7d3247649d4bf4eec6704b1a7c8 P 8b439d6b462a49188ab751501b266fdd: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 15:08:53.820860 10640 ts_tablet_manager.cc:1397] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 15:08:53.821252 10647 raft_consensus.cc:348] T f12bc7d3247649d4bf4eec6704b1a7c8 P 8b439d6b462a49188ab751501b266fdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f34c101cebf4318a7484b40cdfc072a" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45381 } } peers { permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 } } peers { permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 } }
I20260811 15:08:53.821352 10647 raft_consensus.cc:374] T f12bc7d3247649d4bf4eec6704b1a7c8 P 8b439d6b462a49188ab751501b266fdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:53.821382 10647 raft_consensus.cc:729] T f12bc7d3247649d4bf4eec6704b1a7c8 P 8b439d6b462a49188ab751501b266fdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8b439d6b462a49188ab751501b266fdd, State: Initialized, Role: FOLLOWER
I20260811 15:08:53.821328 10640 raft_consensus.cc:348] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f34c101cebf4318a7484b40cdfc072a" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45381 } } peers { permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 } } peers { permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 } }
I20260811 15:08:53.821458 10640 raft_consensus.cc:374] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:53.821530 10640 raft_consensus.cc:729] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f34c101cebf4318a7484b40cdfc072a, State: Initialized, Role: FOLLOWER
I20260811 15:08:53.821512 10647 consensus_queue.cc:260] T f12bc7d3247649d4bf4eec6704b1a7c8 P 8b439d6b462a49188ab751501b266fdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f34c101cebf4318a7484b40cdfc072a" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45381 } } peers { permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 } } peers { permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 } }
I20260811 15:08:53.821671 10640 consensus_queue.cc:260] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f34c101cebf4318a7484b40cdfc072a" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45381 } } peers { permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 } } peers { permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 } }
I20260811 15:08:53.821853 10647 ts_tablet_manager.cc:1428] T f12bc7d3247649d4bf4eec6704b1a7c8 P 8b439d6b462a49188ab751501b266fdd: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:53.822650 10656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.822271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.1:54349)
0811 15:08:53.822492 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.822553 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":102,"threads_started":1}
I20260811 15:08:53.822789 10640 ts_tablet_manager.cc:1428] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 15:08:53.824610 10255 heartbeater.cc:503] Master 127.30.8.61:40131 was elected leader, sending a full tablet report...
I20260811 15:08:53.824637 10654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.824351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.2:56789)
0811 15:08:53.824506 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.824560 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 15:08:53.825937 10658 raft_consensus.cc:482] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:08:53.826005 10046 heartbeater.cc:503] Master 127.30.8.61:40131 was elected leader, sending a full tablet report...
I20260811 15:08:53.826042 10658 raft_consensus.cc:504] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f34c101cebf4318a7484b40cdfc072a" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45381 } } peers { permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 } } peers { permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 } }
I20260811 15:08:53.826241 10656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.825676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.1:43145)
0811 15:08:53.825924 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.825981 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
I20260811 15:08:53.826685 10477 heartbeater.cc:503] Master 127.30.8.61:40131 was elected leader, sending a full tablet report...
I20260811 15:08:53.826712 10181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f12bc7d3247649d4bf4eec6704b1a7c8" candidate_uuid: "0f34c101cebf4318a7484b40cdfc072a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea2d50e2519476984b8fec3e8d172bc" is_pre_election: true
I20260811 15:08:53.826844 10181 raft_consensus.cc:2454] T f12bc7d3247649d4bf4eec6704b1a7c8 P fea2d50e2519476984b8fec3e8d172bc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34c101cebf4318a7484b40cdfc072a in term 0.
I20260811 15:08:53.827137 10294 leader_election.cc:302] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34c101cebf4318a7484b40cdfc072a, fea2d50e2519476984b8fec3e8d172bc; no voters:
I20260811 15:08:53.827472 10657 raft_consensus.cc:2781] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:08:53.827541 10657 raft_consensus.cc:482] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:08:53.827580 10657 raft_consensus.cc:3037] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:53.828811 10657 raft_consensus.cc:504] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f34c101cebf4318a7484b40cdfc072a" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45381 } } peers { permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 } } peers { permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 } }
I20260811 15:08:53.829167 10657 leader_election.cc:288] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [CANDIDATE]: Term 1 election: Requested vote from peers fea2d50e2519476984b8fec3e8d172bc (127.30.8.2:41249), 8b439d6b462a49188ab751501b266fdd (127.30.8.1:37731)
I20260811 15:08:53.829344 10658 leader_election.cc:288] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fea2d50e2519476984b8fec3e8d172bc (127.30.8.2:41249), 8b439d6b462a49188ab751501b266fdd (127.30.8.1:37731)
I20260811 15:08:53.829486 10181 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f12bc7d3247649d4bf4eec6704b1a7c8" candidate_uuid: "0f34c101cebf4318a7484b40cdfc072a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fea2d50e2519476984b8fec3e8d172bc"
I20260811 15:08:53.829618 10181 raft_consensus.cc:3037] T f12bc7d3247649d4bf4eec6704b1a7c8 P fea2d50e2519476984b8fec3e8d172bc [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:53.829700 10659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.823825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.3:47511)
0811 15:08:53.829535 (+ 5710us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.829599 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":5638,"thread_start_us":93,"threads_started":1}
I20260811 15:08:53.829773 9975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f12bc7d3247649d4bf4eec6704b1a7c8" candidate_uuid: "0f34c101cebf4318a7484b40cdfc072a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b439d6b462a49188ab751501b266fdd" is_pre_election: true
I20260811 15:08:53.829908 9975 raft_consensus.cc:2454] T f12bc7d3247649d4bf4eec6704b1a7c8 P 8b439d6b462a49188ab751501b266fdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0f34c101cebf4318a7484b40cdfc072a in term 0.
I20260811 15:08:53.830461 10659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.830214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.3:54975)
0811 15:08:53.830344 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.830397 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260811 15:08:53.831028 10181 raft_consensus.cc:2454] T f12bc7d3247649d4bf4eec6704b1a7c8 P fea2d50e2519476984b8fec3e8d172bc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34c101cebf4318a7484b40cdfc072a in term 1.
I20260811 15:08:53.831274 9978 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f12bc7d3247649d4bf4eec6704b1a7c8" candidate_uuid: "0f34c101cebf4318a7484b40cdfc072a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8b439d6b462a49188ab751501b266fdd"
I20260811 15:08:53.831318 10294 leader_election.cc:302] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0f34c101cebf4318a7484b40cdfc072a, fea2d50e2519476984b8fec3e8d172bc; no voters:
I20260811 15:08:53.831398 9978 raft_consensus.cc:3037] T f12bc7d3247649d4bf4eec6704b1a7c8 P 8b439d6b462a49188ab751501b266fdd [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:53.831472 10658 raft_consensus.cc:2781] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:53.831596 10658 raft_consensus.cc:686] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [term 1 LEADER]: Becoming Leader. State: Replica: 0f34c101cebf4318a7484b40cdfc072a, State: Running, Role: LEADER
I20260811 15:08:53.831748 10658 consensus_queue.cc:237] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f34c101cebf4318a7484b40cdfc072a" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45381 } } peers { permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 } } peers { permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 } }
I20260811 15:08:53.832499 10659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.832081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.3:49485)
0811 15:08:53.832384 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.832434 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242}
I20260811 15:08:53.832666 9978 raft_consensus.cc:2454] T f12bc7d3247649d4bf4eec6704b1a7c8 P 8b439d6b462a49188ab751501b266fdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0f34c101cebf4318a7484b40cdfc072a in term 1.
I20260811 15:08:53.833570 9468 catalog_manager.cc:5495] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a reported cstate change: term changed from 0 to 1, leader changed from <none> to 0f34c101cebf4318a7484b40cdfc072a (127.30.8.3). New cstate: current_term: 1 leader_uuid: "0f34c101cebf4318a7484b40cdfc072a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f34c101cebf4318a7484b40cdfc072a" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 45381 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 } health_report { overall_health: UNKNOWN } } }
I20260811 15:08:53.853322 10629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.850326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44634 (local address 127.30.8.3:45381)
0811 15:08:53.850450 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:53.850455 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:53.850471 (+ 16us) server_negotiation.cc:407] Connection header received
0811 15:08:53.850590 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:53.850594 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:53.850645 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:53.850705 (+ 60us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:53.850997 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:53.851530 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:53.852459 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:53.852675 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:53.852761 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:53.852941 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:53.853016 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:53.853063 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:53.853135 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 15:08:53.853188 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 15:08:53.855640 9978 raft_consensus.cc:1261] T f12bc7d3247649d4bf4eec6704b1a7c8 P 8b439d6b462a49188ab751501b266fdd [term 1 FOLLOWER]: Refusing update from remote peer 0f34c101cebf4318a7484b40cdfc072a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:53.855659 10181 raft_consensus.cc:1261] T f12bc7d3247649d4bf4eec6704b1a7c8 P fea2d50e2519476984b8fec3e8d172bc [term 1 FOLLOWER]: Refusing update from remote peer 0f34c101cebf4318a7484b40cdfc072a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:53.856129 10658 consensus_queue.cc:1035] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fea2d50e2519476984b8fec3e8d172bc" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:53.856297 10657 consensus_queue.cc:1035] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8b439d6b462a49188ab751501b266fdd" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 37731 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:53.858260 10654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.857941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.2:37631)
0811 15:08:53.858097 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.858159 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260811 15:08:53.860244 10666 mvcc.cc:205] Tried to move back new op lower bound from 7317343985068048384 to 7317343984976191488. Current Snapshot: MvccSnapshot[applied={T|T < 7317343985068048384}]
I20260811 15:08:53.860548 10656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.857520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.1:50563)
0811 15:08:53.860398 (+ 2878us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.860462 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2793}
I20260811 15:08:53.861590 10659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.861359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.3:46399)
0811 15:08:53.861455 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.861513 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260811 15:08:53.862820 10659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.862616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.3:57663)
0811 15:08:53.862707 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.862757 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 15:08:53.863488 10654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.863231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.2:50111)
0811 15:08:53.863364 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.863416 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 15:08:53.865546 10656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.862312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:36293 (local address 127.30.8.1:38199)
0811 15:08:53.865412 (+ 3100us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.865470 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:36293: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3025}
I20260811 15:08:53.868242 9469 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:53.868392 9469 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52424:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:53.872123 9469 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd9736da73324408b58ba279deb3f85e: Sending DeleteTablet for 3 replicas of tablet f12bc7d3247649d4bf4eec6704b1a7c8
I20260811 15:08:53.872814 10371 tablet_service.cc:1514] Processing DeleteTablet for tablet f12bc7d3247649d4bf4eec6704b1a7c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:53 UTC) from {username='oryx', principal='oryx/127.30.8.61@KRBTEST.COM'} at 127.0.0.1:44624
I20260811 15:08:53.872890 10153 tablet_service.cc:1514] Processing DeleteTablet for tablet f12bc7d3247649d4bf4eec6704b1a7c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:53 UTC) from {username='oryx', principal='oryx/127.30.8.61@KRBTEST.COM'} at 127.0.0.1:51430
I20260811 15:08:53.872996 9952 tablet_service.cc:1514] Processing DeleteTablet for tablet f12bc7d3247649d4bf4eec6704b1a7c8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:53 UTC) from {username='oryx', principal='oryx/127.30.8.61@KRBTEST.COM'} at 127.0.0.1:34314
I20260811 15:08:53.873425 10677 tablet_replica.cc:330] T f12bc7d3247649d4bf4eec6704b1a7c8 P fea2d50e2519476984b8fec3e8d172bc: stopping tablet replica
I20260811 15:08:53.873601 10677 raft_consensus.cc:2229] T f12bc7d3247649d4bf4eec6704b1a7c8 P fea2d50e2519476984b8fec3e8d172bc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:53.873792 10677 raft_consensus.cc:2258] T f12bc7d3247649d4bf4eec6704b1a7c8 P fea2d50e2519476984b8fec3e8d172bc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:53.873816 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 9713
I20260811 15:08:53.886709 10675 tablet_replica.cc:330] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a: stopping tablet replica
I20260811 15:08:53.887022 10675 raft_consensus.cc:2229] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:08:53.887406 10675 raft_consensus.cc:2258] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:53.887440 10677 ts_tablet_manager.cc:1905] T f12bc7d3247649d4bf4eec6704b1a7c8 P fea2d50e2519476984b8fec3e8d172bc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:53.888386 10675 ts_tablet_manager.cc:1905] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:53.889896 10677 ts_tablet_manager.cc:1918] T f12bc7d3247649d4bf4eec6704b1a7c8 P fea2d50e2519476984b8fec3e8d172bc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:08:53.889991 10677 log.cc:1196] T f12bc7d3247649d4bf4eec6704b1a7c8 P fea2d50e2519476984b8fec3e8d172bc: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/f12bc7d3247649d4bf4eec6704b1a7c8
I20260811 15:08:53.890282 10677 ts_tablet_manager.cc:1939] T f12bc7d3247649d4bf4eec6704b1a7c8 P fea2d50e2519476984b8fec3e8d172bc: Deleting consensus metadata
I20260811 15:08:53.891201 9446 catalog_manager.cc:4841] TS fea2d50e2519476984b8fec3e8d172bc (127.30.8.2:41249): tablet f12bc7d3247649d4bf4eec6704b1a7c8 (table test-table [id=3457d3edcb444e598d0a2cd2093d571e]) successfully deleted
I20260811 15:08:53.891943 10675 ts_tablet_manager.cc:1918] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:08:53.892051 10675 log.cc:1196] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/f12bc7d3247649d4bf4eec6704b1a7c8
I20260811 15:08:53.892360 10675 ts_tablet_manager.cc:1939] T f12bc7d3247649d4bf4eec6704b1a7c8 P 0f34c101cebf4318a7484b40cdfc072a: Deleting consensus metadata
I20260811 15:08:53.893298 9450 catalog_manager.cc:4841] TS 0f34c101cebf4318a7484b40cdfc072a (127.30.8.3:45381): tablet f12bc7d3247649d4bf4eec6704b1a7c8 (table test-table [id=3457d3edcb444e598d0a2cd2093d571e]) successfully deleted
I20260811 15:08:53.895568 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 10053
I20260811 15:08:53.896003 10637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:53.895558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:37731 (local address 127.0.0.1:34320)
0811 15:08:53.895797 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:53.895881 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.1:37731: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
W20260811 15:08:53.907847 9444 catalog_manager.cc:4584] TS 8b439d6b462a49188ab751501b266fdd (127.30.8.1:37731): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f12bc7d3247649d4bf4eec6704b1a7c8: Network error: Client connection negotiation failed: client connection to 127.30.8.1:37731: connect: Connection refused (error 111)
I20260811 15:08:53.917505 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 10262
I20260811 15:08:53.935286 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 9234
I20260811 15:08:53.959144 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 9414
2026-08-11T15:08:53Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (7862 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:08:54 dist-test-slave-1vmg krb5kdc[10682](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:08:54 dist-test-slave-1vmg krb5kdc[10682](info): set up 2 sockets
Aug 11 15:08:54 dist-test-slave-1vmg krb5kdc[10682](info): commencing operation
krb5kdc: starting...
W20260811 15:08:55.156435 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:08:55 dist-test-slave-1vmg krb5kdc[10682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:08:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:08:55Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.62.keytab.
Entry for principal oryx/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.62.keytab.
I20260811 15:08:55.305917 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:44939
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.62.keytab
--principal=oryx/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:44939
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:55.431102 10707 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:55.431447 10707 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:55.431519 10707 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:55.435817 10707 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:08:55.435915 10707 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 15:08:55.435946 10707 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:55.435976 10707 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:55.436028 10707 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:08:55.436066 10707 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:08:55.441938 10707 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:44939
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.62.keytab
--principal=oryx/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:44939
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10707
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:55.443339 10707 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:55.444463 10707 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:55.454210 10714 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:55.454267 10707 server_base.cc:1034] running on GCE node
W20260811 15:08:55.454214 10713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:55.454214 10716 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:55.455437 10707 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:55.456710 10707 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:55.457901 10707 hybrid_clock.cc:648] HybridClock initialized: now 1786460935457887 us; error 51 us; skew 500 ppm
Aug 11 15:08:55 dist-test-slave-1vmg krb5kdc[10682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:55.466316 10707 init.cc:377] Logged in from keytab as oryx/127.30.8.62@KRBTEST.COM (short username oryx)
I20260811 15:08:55.467828 10707 webserver.cc:460] Webserver started at http://127.30.8.62:37933/ using document root <none> and password file <none>
I20260811 15:08:55.468595 10707 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:55.468672 10707 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:55.469129 10707 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:55.471737 10707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "bd7b8f4a58ca4ed8bea296e790287e49"
format_stamp: "Formatted at 2026-08-11 15:08:55 on dist-test-slave-1vmg"
server_key: "55469768b23158cbd077e07363db28a1"
server_key_iv: "05b74f055eab43a087bec9a264bc5a74"
server_key_version: "encryptionkey@0"
I20260811 15:08:55.472359 10707 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "bd7b8f4a58ca4ed8bea296e790287e49"
format_stamp: "Formatted at 2026-08-11 15:08:55 on dist-test-slave-1vmg"
server_key: "55469768b23158cbd077e07363db28a1"
server_key_iv: "05b74f055eab43a087bec9a264bc5a74"
server_key_version: "encryptionkey@0"
I20260811 15:08:55.476989 10707 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:08:55.480520 10729 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:55.482004 10707 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 15:08:55.482156 10707 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "bd7b8f4a58ca4ed8bea296e790287e49"
format_stamp: "Formatted at 2026-08-11 15:08:55 on dist-test-slave-1vmg"
server_key: "55469768b23158cbd077e07363db28a1"
server_key_iv: "05b74f055eab43a087bec9a264bc5a74"
server_key_version: "encryptionkey@0"
I20260811 15:08:55.482277 10707 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:55.503670 10707 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:55.507839 10707 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:55.508095 10707 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:55.516825 10707 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:44939
I20260811 15:08:55.516837 10825 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:44939 every 8 connection(s)
I20260811 15:08:55.517943 10707 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:08:55.521206 10826 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:55.525363 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 10707
I20260811 15:08:55.525511 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:08:55.525836 30752 external_mini_cluster.cc:1428] Setting key 7f6cbd42981b72e1fa5dca5949f1028b
I20260811 15:08:55.528295 10826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49: Bootstrap starting.
I20260811 15:08:55.530959 10826 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:55.531818 10826 log.cc:824] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:55.534008 10826 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49: No bootstrap required, opened a new log
I20260811 15:08:55.537943 10826 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b8f4a58ca4ed8bea296e790287e49" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44939 } }
I20260811 15:08:55.538246 10826 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:55.538302 10826 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd7b8f4a58ca4ed8bea296e790287e49, State: Initialized, Role: FOLLOWER
I20260811 15:08:55.538816 10826 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b8f4a58ca4ed8bea296e790287e49" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44939 } }
I20260811 15:08:55.539011 10826 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:08:55.539095 10826 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:08:55.539206 10826 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:55.540292 10826 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b8f4a58ca4ed8bea296e790287e49" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44939 } }
I20260811 15:08:55.540693 10826 leader_election.cc:302] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd7b8f4a58ca4ed8bea296e790287e49; no voters:
Aug 11 15:08:55 dist-test-slave-1vmg krb5kdc[10682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460935, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM
I20260811 15:08:55.541162 10826 leader_election.cc:288] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:08:55.541435 10832 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:55.541769 10832 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49 [term 1 LEADER]: Becoming Leader. State: Replica: bd7b8f4a58ca4ed8bea296e790287e49, State: Running, Role: LEADER
I20260811 15:08:55.542178 10832 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b8f4a58ca4ed8bea296e790287e49" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44939 } }
I20260811 15:08:55.542739 10826 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:08:55.544194 10833 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd7b8f4a58ca4ed8bea296e790287e49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b8f4a58ca4ed8bea296e790287e49" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44939 } } }
I20260811 15:08:55.544340 10833 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:55.544826 10834 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd7b8f4a58ca4ed8bea296e790287e49. Latest consensus state: current_term: 1 leader_uuid: "bd7b8f4a58ca4ed8bea296e790287e49" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd7b8f4a58ca4ed8bea296e790287e49" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44939 } } }
I20260811 15:08:55.545019 10834 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49 [sys.catalog]: This master's current role is: LEADER
I20260811 15:08:55.548435 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:55.527520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54282 (local address 127.30.8.62:44939)
0811 15:08:55.528285 (+ 765us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:55.528327 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:55.528424 (+ 97us) server_negotiation.cc:407] Connection header received
0811 15:08:55.529398 (+ 974us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:55.529424 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:55.529933 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:55.530395 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:55.531544 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:55.532749 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:55.533684 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:55.534298 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:55.534473 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:55.542053 (+ 7580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:55.542060 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:55.542083 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:55.542134 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:55.544654 (+ 2520us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:55.545380 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:55.545387 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:55.545394 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:55.545550 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:55.545913 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:55.545917 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:55.545918 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:55.546477 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:55.546654 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:55.547049 (+ 395us) server_negotiation.cc:299] Negotiation successful
0811 15:08:55.547428 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":167,"threads_started":1}
W20260811 15:08:55.548780 10854 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:08:55.548947 10854 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:08:55.549072 10847 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:08:55.550055 10847 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:08:55.556758 10847 catalog_manager.cc:1347] Generated new cluster ID: ed066ea3c6464aaeb10deae8371072c6
I20260811 15:08:55.556898 10847 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:08:55.567665 10847 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:08:55.569653 10847 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:08:55.578197 10847 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49: Generated new TSK 0
I20260811 15:08:55.579105 10847 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.1.keytab.
Entry for principal oryx/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.1.keytab.
I20260811 15:08:55.626137 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:44939
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.1.keytab
--principal=oryx/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:55.785023 10862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:55.785379 10862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:55.785431 10862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:55.789896 10862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:55.789996 10862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:55.790043 10862 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 15:08:55.790096 10862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:08:55.796226 10862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.1.keytab
--principal=oryx/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.8.62:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:55.797550 10862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:55.798703 10862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 15:08:55.809777 10862 server_base.cc:1034] running on GCE node
W20260811 15:08:55.809731 10867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:55.809732 10868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:55.809731 10871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:55.810611 10862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:55.811408 10862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:55.812587 10862 hybrid_clock.cc:648] HybridClock initialized: now 1786460935812564 us; error 38 us; skew 500 ppm
Aug 11 15:08:55 dist-test-slave-1vmg krb5kdc[10682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:55.815779 10862 init.cc:377] Logged in from keytab as oryx/127.30.8.1@KRBTEST.COM (short username oryx)
I20260811 15:08:55.816848 10862 webserver.cc:460] Webserver started at http://127.30.8.1:41539/ using document root <none> and password file <none>
I20260811 15:08:55.817507 10862 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:55.817584 10862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:55.817807 10862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:55.820230 10862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "88cf1c4770f64b97aef9328095fa61ce"
format_stamp: "Formatted at 2026-08-11 15:08:55 on dist-test-slave-1vmg"
server_key: "36a24e5f8d36c706acf742848a68a52d"
server_key_iv: "b27b3381e8082aed214a1e14e83ba357"
server_key_version: "encryptionkey@0"
I20260811 15:08:55.820832 10862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "88cf1c4770f64b97aef9328095fa61ce"
format_stamp: "Formatted at 2026-08-11 15:08:55 on dist-test-slave-1vmg"
server_key: "36a24e5f8d36c706acf742848a68a52d"
server_key_iv: "b27b3381e8082aed214a1e14e83ba357"
server_key_version: "encryptionkey@0"
I20260811 15:08:55.825554 10862 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:08:55.828518 10887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:55.829941 10862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 15:08:55.830078 10862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "88cf1c4770f64b97aef9328095fa61ce"
format_stamp: "Formatted at 2026-08-11 15:08:55 on dist-test-slave-1vmg"
server_key: "36a24e5f8d36c706acf742848a68a52d"
server_key_iv: "b27b3381e8082aed214a1e14e83ba357"
server_key_version: "encryptionkey@0"
I20260811 15:08:55.830209 10862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:55.860175 10862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:55.863721 10862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:55.863934 10862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:55.865553 10862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:55.865613 10862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:55.865672 10862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:55.865703 10862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 15:08:55 dist-test-slave-1vmg krb5kdc[10682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM
I20260811 15:08:55.876542 10862 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:44543
I20260811 15:08:55.876976 11061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:44543 every 8 connection(s)
I20260811 15:08:55.877648 10862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:08:55.880049 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:55.866644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:51317 (local address 127.30.8.62:44939)
0811 15:08:55.866804 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:55.866809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:55.867811 (+ 1002us) server_negotiation.cc:407] Connection header received
0811 15:08:55.869169 (+ 1358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:55.869173 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:55.869250 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:55.869350 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:55.870892 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:55.871512 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:55.872806 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:55.873058 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:55.873151 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:55.876047 (+ 2896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:55.876051 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:55.876055 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:55.876087 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:55.877606 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:55.878253 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:55.878257 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:55.878259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:55.878357 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:55.878755 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:55.878759 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:55.878761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:55.879049 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:55.879158 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:55.879706 (+ 548us) server_negotiation.cc:299] Negotiation successful
0811 15:08:55.879880 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 15:08:55.880673 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 10862
I20260811 15:08:55.880841 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:08:55.881024 10898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:55.867084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:44939 (local address 127.30.8.1:51317)
0811 15:08:55.867632 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:55.867691 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:55.868897 (+ 1206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:55.869553 (+ 656us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:55.869561 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:55.870025 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:55.870725 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:55.870743 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:55.871646 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:55.871650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:55.872673 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:55.872693 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:55.873229 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:55.873233 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:55.873415 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:55.874072 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:55.874091 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:55.875821 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:55.877740 (+ 1919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:55.877747 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:55.877763 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:55.878116 (+ 353us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:55.878464 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:55.878468 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:55.878471 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:55.878646 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:55.879162 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:55.879170 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:55.879446 (+ 276us) client_negotiation.cc:769] Sending connection context
0811 15:08:55.879651 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 15:08:55.880034 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":120,"threads_started":1}
I20260811 15:08:55.881222 30752 external_mini_cluster.cc:1428] Setting key 1c886475a71ced2c86dd68aea0428f07
I20260811 15:08:55.882635 11062 heartbeater.cc:347] Connected to a master server at 127.30.8.62:44939
I20260811 15:08:55.882874 11062 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:55.883500 11062 heartbeater.cc:511] Master 127.30.8.62:44939 requested a full tablet report, sending...
I20260811 15:08:55.885213 10757 ts_manager.cc:194] Registered new tserver with Master: 88cf1c4770f64b97aef9328095fa61ce (127.30.8.1:44543)
I20260811 15:08:55.886729 10757 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:51317
I20260811 15:08:55.894094 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:55.886340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:42357 (local address 127.30.8.62:44939)
0811 15:08:55.886492 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:55.886497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:55.886791 (+ 294us) server_negotiation.cc:407] Connection header received
0811 15:08:55.886962 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:55.886965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:55.887016 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:55.887089 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:55.887714 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:55.888276 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:55.889339 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:55.889560 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:55.889664 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:55.890900 (+ 1236us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:55.890904 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:55.890907 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:55.890959 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:55.892459 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:55.892878 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:55.892881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:55.892883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:55.892966 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:55.893235 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:55.893239 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:55.893241 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:55.893436 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:55.893526 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:55.893798 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 15:08:55.893918 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 15:08:55.894133 11074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:55.886266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:44939 (local address 127.30.8.1:42357)
0811 15:08:55.886690 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:55.886703 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:55.886838 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:55.887130 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:55.887133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:55.887394 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:55.887551 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:55.887561 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:55.888457 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:55.888462 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:55.889202 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:55.889213 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:55.889680 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:55.889683 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:55.889782 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:55.890330 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:55.890343 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:55.890749 (+ 406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:55.892574 (+ 1825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:55.892578 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:55.892580 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:55.892781 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:55.893052 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:55.893055 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:55.893057 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:55.893149 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:55.893530 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:55.893534 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:55.893636 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 15:08:55.893727 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 15:08:55.893902 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"mutex_wait_us":76,"thread_start_us":144,"threads_started":1}
WARNING: no policy specified for oryx/127.30.8.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.8.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.2.keytab.
Entry for principal oryx/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.2.keytab.
I20260811 15:08:55.918648 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:44939
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.2.keytab
--principal=oryx/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:56.050252 11077 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:56.050724 11077 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:56.050800 11077 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:56.055429 11077 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:56.055533 11077 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:56.055580 11077 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 15:08:56.055617 11077 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:08:56.061985 11077 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.2.keytab
--principal=oryx/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.8.62:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11077
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:56.063525 11077 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:56.064849 11077 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 15:08:56.076886 11077 server_base.cc:1034] running on GCE node
W20260811 15:08:56.076838 11086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:56.076838 11085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:56.076838 11088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:56.077636 11077 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:56.078400 11077 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:56.079558 11077 hybrid_clock.cc:648] HybridClock initialized: now 1786460936079540 us; error 33 us; skew 500 ppm
Aug 11 15:08:56 dist-test-slave-1vmg krb5kdc[10682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460936, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:56.082717 11077 init.cc:377] Logged in from keytab as oryx/127.30.8.2@KRBTEST.COM (short username oryx)
I20260811 15:08:56.083865 11077 webserver.cc:460] Webserver started at http://127.30.8.2:35855/ using document root <none> and password file <none>
I20260811 15:08:56.084506 11077 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:56.084589 11077 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:56.084836 11077 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:56.087271 11077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "982a70d4b9d44b51925104fde0947838"
format_stamp: "Formatted at 2026-08-11 15:08:56 on dist-test-slave-1vmg"
server_key: "60d34e2b1265fd96d86196895e45af87"
server_key_iv: "a695d4ed7803693d44ff92ddb7bf0c88"
server_key_version: "encryptionkey@0"
I20260811 15:08:56.087903 11077 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "982a70d4b9d44b51925104fde0947838"
format_stamp: "Formatted at 2026-08-11 15:08:56 on dist-test-slave-1vmg"
server_key: "60d34e2b1265fd96d86196895e45af87"
server_key_iv: "a695d4ed7803693d44ff92ddb7bf0c88"
server_key_version: "encryptionkey@0"
I20260811 15:08:56.092141 11077 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 15:08:56.095060 11102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:56.096427 11077 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:08:56.096572 11077 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "982a70d4b9d44b51925104fde0947838"
format_stamp: "Formatted at 2026-08-11 15:08:56 on dist-test-slave-1vmg"
server_key: "60d34e2b1265fd96d86196895e45af87"
server_key_iv: "a695d4ed7803693d44ff92ddb7bf0c88"
server_key_version: "encryptionkey@0"
I20260811 15:08:56.096688 11077 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:56.112506 11077 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:56.116704 11077 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:56.117009 11077 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:56.119127 11077 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:56.119202 11077 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:56.119270 11077 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:56.119302 11077 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 15:08:56 dist-test-slave-1vmg krb5kdc[10682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460936, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM
I20260811 15:08:56.132634 11077 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:35591
I20260811 15:08:56.132670 11285 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:35591 every 8 connection(s)
I20260811 15:08:56.133875 11077 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:08:56.134791 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.120130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:38751 (local address 127.30.8.62:44939)
0811 15:08:56.120262 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:56.120265 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:56.121765 (+ 1500us) server_negotiation.cc:407] Connection header received
0811 15:08:56.122967 (+ 1202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:56.122971 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:56.123031 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:56.123105 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:56.125028 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.125592 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.126984 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.127212 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.127306 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.130729 (+ 3423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:56.130733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:56.130736 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:56.130768 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:56.132333 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:56.133021 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:56.133024 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:56.133026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:56.133118 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:56.133469 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:56.133472 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:56.133474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:56.133711 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:56.133805 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:56.134464 (+ 659us) server_negotiation.cc:299] Negotiation successful
0811 15:08:56.134612 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 15:08:56.135895 11115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.120893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:44939 (local address 127.30.8.2:38751)
0811 15:08:56.121573 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:56.121632 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:56.122643 (+ 1011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:56.123579 (+ 936us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:56.123590 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:56.124150 (+ 560us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:56.124853 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.124869 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.125729 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.125733 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.126838 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.126850 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.127393 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.127398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.127632 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.128570 (+ 938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:56.128594 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:56.130523 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:56.132485 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:56.132492 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:56.132513 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:56.132918 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:56.133210 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:56.133213 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:56.133215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:56.133370 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:56.133824 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:56.133832 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:56.134206 (+ 374us) client_negotiation.cc:769] Sending connection context
0811 15:08:56.134401 (+ 195us) client_negotiation.cc:240] Negotiation successful
0811 15:08:56.134819 (+ 418us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":179,"threads_started":1}
I20260811 15:08:56.137308 11287 heartbeater.cc:347] Connected to a master server at 127.30.8.62:44939
I20260811 15:08:56.137482 11287 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:56.137990 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 11077
I20260811 15:08:56.138114 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:08:56.138172 11287 heartbeater.cc:511] Master 127.30.8.62:44939 requested a full tablet report, sending...
I20260811 15:08:56.138456 30752 external_mini_cluster.cc:1428] Setting key 4af96401384fd7bcf24bbca3746f85ad
I20260811 15:08:56.139451 10755 ts_manager.cc:194] Registered new tserver with Master: 982a70d4b9d44b51925104fde0947838 (127.30.8.2:35591)
I20260811 15:08:56.140193 10755 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:38751
I20260811 15:08:56.149492 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.140517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:56165 (local address 127.30.8.62:44939)
0811 15:08:56.140663 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:56.140667 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:56.140681 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:08:56.140717 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:56.140720 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:56.140761 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:56.140842 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:56.143281 (+ 2439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.143807 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.144679 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.144898 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.144984 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.146348 (+ 1364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:56.146351 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:56.146353 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:56.146377 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:56.147792 (+ 1415us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:56.148297 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:56.148299 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:56.148300 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:56.148375 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:56.148643 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:56.148645 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:56.148646 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:56.148870 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:56.148946 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:56.149255 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 15:08:56.149359 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 15:08:56.149587 11294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.139699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:44939 (local address 127.30.8.2:56165)
0811 15:08:56.140018 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:56.140034 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:56.140140 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:56.142646 (+ 2506us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:56.142649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:56.142958 (+ 309us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:56.143110 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.143118 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.143954 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.143957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.144565 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.144575 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.145033 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.145037 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.145138 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.145662 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:56.145677 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:56.146153 (+ 476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:56.147917 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:56.147921 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:56.147923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:56.148201 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:56.148458 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:56.148462 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:56.148464 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:56.148564 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:56.148965 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:56.148968 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:56.149082 (+ 114us) client_negotiation.cc:769] Sending connection context
0811 15:08:56.149209 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 15:08:56.149375 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":108,"threads_started":1}
WARNING: no policy specified for oryx/127.30.8.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.8.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.3.keytab.
Entry for principal oryx/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.3.keytab.
I20260811 15:08:56.174448 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:44939
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.3.keytab
--principal=oryx/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:36807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:56.302968 11299 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:56.303310 11299 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:56.303365 11299 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:56.308013 11299 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:56.308104 11299 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:56.308148 11299 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 15:08:56.308184 11299 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:08:56.314052 11299 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.3.keytab
--principal=oryx/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.8.62:44939
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11299
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:56.315327 11299 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:56.316440 11299 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:56.327414 11312 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:56.327394 11306 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:56.327721 11299 server_base.cc:1034] running on GCE node
W20260811 15:08:56.327447 11310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:56.328629 11299 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:56.329495 11299 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:56.330694 11299 hybrid_clock.cc:648] HybridClock initialized: now 1786460936330655 us; error 58 us; skew 500 ppm
Aug 11 15:08:56 dist-test-slave-1vmg krb5kdc[10682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460936, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:08:56.333963 11299 init.cc:377] Logged in from keytab as oryx/127.30.8.3@KRBTEST.COM (short username oryx)
I20260811 15:08:56.335109 11299 webserver.cc:460] Webserver started at http://127.30.8.3:35685/ using document root <none> and password file <none>
I20260811 15:08:56.335716 11299 fs_manager.cc:359] Metadata directory not provided
I20260811 15:08:56.335780 11299 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:08:56.336002 11299 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:08:56.338292 11299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "79314a19dd6643c286c38475262d781f"
format_stamp: "Formatted at 2026-08-11 15:08:56 on dist-test-slave-1vmg"
server_key: "ecb53c1d8e16c3fc30b4125c5d2df9a8"
server_key_iv: "1e323bafd6d5089b958adec8dead47b4"
server_key_version: "encryptionkey@0"
I20260811 15:08:56.338876 11299 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "79314a19dd6643c286c38475262d781f"
format_stamp: "Formatted at 2026-08-11 15:08:56 on dist-test-slave-1vmg"
server_key: "ecb53c1d8e16c3fc30b4125c5d2df9a8"
server_key_iv: "1e323bafd6d5089b958adec8dead47b4"
server_key_version: "encryptionkey@0"
I20260811 15:08:56.342942 11299 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 15:08:56.345741 11319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:56.347070 11299 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:08:56.347194 11299 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "79314a19dd6643c286c38475262d781f"
format_stamp: "Formatted at 2026-08-11 15:08:56 on dist-test-slave-1vmg"
server_key: "ecb53c1d8e16c3fc30b4125c5d2df9a8"
server_key_iv: "1e323bafd6d5089b958adec8dead47b4"
server_key_version: "encryptionkey@0"
I20260811 15:08:56.347303 11299 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:08:56.365630 11299 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:08:56.369271 11299 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:08:56.369472 11299 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:08:56.371152 11299 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:08:56.371217 11299 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:56.371263 11299 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:08:56.371292 11299 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:08:56.382266 11299 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:44281
I20260811 15:08:56.382342 11483 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:44281 every 8 connection(s)
Aug 11 15:08:56 dist-test-slave-1vmg krb5kdc[10682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460936, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM
I20260811 15:08:56.383498 11299 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:08:56.384456 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 11299
I20260811 15:08:56.384590 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:08:56.384881 30752 external_mini_cluster.cc:1428] Setting key c69f1637a43ce9d61a9e38767707d382
I20260811 15:08:56.387223 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.372047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:39935 (local address 127.30.8.62:44939)
0811 15:08:56.372181 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:56.372185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:56.373561 (+ 1376us) server_negotiation.cc:407] Connection header received
0811 15:08:56.374790 (+ 1229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:56.374794 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:56.374862 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:56.375011 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:56.376905 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.377753 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.379543 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.379788 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.379876 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.383233 (+ 3357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:56.383238 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:56.383241 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:56.383280 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:56.384947 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:56.385478 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:56.385481 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:56.385482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:56.385572 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:56.385957 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:56.385960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:56.385962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:56.386213 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:56.386309 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:56.386899 (+ 590us) server_negotiation.cc:299] Negotiation successful
0811 15:08:56.387061 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 15:08:56.388183 11335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.372848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:44939 (local address 127.30.8.3:39935)
0811 15:08:56.373385 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:56.373430 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:56.374504 (+ 1074us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:56.375351 (+ 847us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:56.375360 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:56.375893 (+ 533us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:56.376697 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.376721 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.377897 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.377901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.379402 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.379413 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.379961 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.379966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.380196 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.380990 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:56.381014 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:56.382992 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:56.385074 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:56.385080 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:56.385097 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:56.385368 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:56.385670 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:56.385673 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:56.385674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:56.385861 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:56.386308 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:56.386316 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:56.386658 (+ 342us) client_negotiation.cc:769] Sending connection context
0811 15:08:56.386837 (+ 179us) client_negotiation.cc:240] Negotiation successful
0811 15:08:56.387245 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":160,"threads_started":1}
I20260811 15:08:56.389343 11484 heartbeater.cc:347] Connected to a master server at 127.30.8.62:44939
I20260811 15:08:56.389503 11484 heartbeater.cc:464] Registering TS with master...
I20260811 15:08:56.390049 11484 heartbeater.cc:511] Master 127.30.8.62:44939 requested a full tablet report, sending...
I20260811 15:08:56.391243 10757 ts_manager.cc:194] Registered new tserver with Master: 79314a19dd6643c286c38475262d781f (127.30.8.3:44281)
I20260811 15:08:56.391999 10757 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:39935
I20260811 15:08:56.398378 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.391652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:53879 (local address 127.30.8.62:44939)
0811 15:08:56.392283 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:56.392286 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:56.392297 (+ 11us) server_negotiation.cc:407] Connection header received
0811 15:08:56.392324 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:56.392326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:56.392355 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:56.392421 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:56.392905 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.393407 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.394205 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.394422 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.394509 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.395629 (+ 1120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:56.395633 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:56.395636 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:56.395662 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:56.396729 (+ 1067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:56.397132 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:56.397136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:56.397138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:56.397218 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:56.397474 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:56.397477 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:56.397479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:56.397675 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:56.397766 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:56.398120 (+ 354us) server_negotiation.cc:299] Negotiation successful
0811 15:08:56.398252 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":576}
I20260811 15:08:56.398422 11496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.391535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:44939 (local address 127.30.8.3:53879)
0811 15:08:56.391844 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:56.391859 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:56.391951 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:56.392470 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:56.392473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:56.392677 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:56.392791 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.392797 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.393533 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.393535 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.394093 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.394101 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.394526 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.394530 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.394616 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.395127 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:56.395138 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:56.395495 (+ 357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:56.396837 (+ 1342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:56.396840 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:56.396842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:56.397040 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:56.397304 (+ 264us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:56.397307 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:56.397309 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:56.397391 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:56.397792 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:56.397796 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:56.397899 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 15:08:56.398043 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 15:08:56.398209 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"mutex_wait_us":1,"thread_start_us":114,"threads_started":1}
I20260811 15:08:56.398896 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 15:08:56 dist-test-slave-1vmg krb5kdc[10682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460936, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 15:08:56 dist-test-slave-1vmg krb5kdc[10682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:08:56 dist-test-slave-1vmg krb5kdc[10682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 15:08:56.425062 10830 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:08:56.418965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54290 (local address 127.30.8.62:44939)
0811 15:08:56.419096 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:56.419100 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:56.419112 (+ 12us) server_negotiation.cc:407] Connection header received
0811 15:08:56.419157 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:56.419160 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:56.419201 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:56.419269 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:56.419993 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.420522 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.421307 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.421509 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.421575 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.424915 (+ 3340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54290: BlockingRecv error: recv got EOF from 127.0.0.1:54290 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 15:08:56 dist-test-slave-1vmg krb5kdc[10682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460936, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM
I20260811 15:08:56.434523 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.426478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54298 (local address 127.30.8.62:44939)
0811 15:08:56.426613 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:56.426617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:56.426805 (+ 188us) server_negotiation.cc:407] Connection header received
0811 15:08:56.426908 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:56.426933 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:56.426984 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:56.427052 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:56.427591 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.428110 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.428895 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.429235 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.429346 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.431746 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:56.431750 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:56.431752 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:56.431780 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:56.433088 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:56.433421 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:56.433423 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:56.433424 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:56.433495 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:56.433682 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:56.433684 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:56.433685 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:56.433901 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:56.434037 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:56.434253 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 15:08:56.434357 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 15:08:56.437603 10757 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54298:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:08:56.440960 10757 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:08:56.457804 11523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.450993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:44281 (local address 127.0.0.1:53276)
0811 15:08:56.452131 (+ 1138us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:56.452203 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:56.452372 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:56.453299 (+ 927us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:56.453302 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:56.453321 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:56.453705 (+ 384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.453712 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.454781 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.454784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.456231 (+ 1447us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.456242 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.457137 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.457141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.457265 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.457310 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 15:08:56.457441 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 15:08:56.457611 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":920,"mutex_wait_us":321,"spinlock_wait_cycles":30592,"thread_start_us":142,"threads_started":1}
I20260811 15:08:56.457811 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.451159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53276 (local address 127.30.8.3:44281)
0811 15:08:56.452720 (+ 1561us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:56.452732 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:56.452776 (+ 44us) server_negotiation.cc:407] Connection header received
0811 15:08:56.452875 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:56.452887 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:56.453135 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:56.453513 (+ 378us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:56.453826 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.454628 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.456369 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.456999 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.457111 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.457352 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:56.457484 (+ 132us) server_negotiation.cc:299] Negotiation successful
0811 15:08:56.457610 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1446,"thread_start_us":163,"threads_started":1}
I20260811 15:08:56.459363 11529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.451812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34838 (local address 127.30.8.2:35591)
0811 15:08:56.452683 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:56.452695 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:56.452718 (+ 23us) server_negotiation.cc:407] Connection header received
0811 15:08:56.452795 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:56.452800 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:56.453020 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:56.453210 (+ 190us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:56.453661 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.455000 (+ 1339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.456454 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.457363 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.457479 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.457688 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:56.459056 (+ 1368us) server_negotiation.cc:299] Negotiation successful
0811 15:08:56.459188 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":757,"thread_start_us":126,"threads_started":1}
I20260811 15:08:56.459566 11530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.452235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42314 (local address 127.30.8.1:44543)
0811 15:08:56.452712 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:56.452720 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:56.452984 (+ 264us) server_negotiation.cc:407] Connection header received
0811 15:08:56.453079 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:56.453086 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:56.453289 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:56.453432 (+ 143us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:56.453841 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.455000 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.457562 (+ 2562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.458508 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.458633 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.458800 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:56.459305 (+ 505us) server_negotiation.cc:299] Negotiation successful
0811 15:08:56.459414 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":97,"threads_started":1}
I20260811 15:08:56.459702 11526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.450783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:35591 (local address 127.0.0.1:34838)
0811 15:08:56.452131 (+ 1348us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:56.452203 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:56.452372 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:56.453212 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:56.453220 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:56.453244 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:56.453520 (+ 276us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.453531 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.455147 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.455151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.456315 (+ 1164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.456327 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.457758 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.457762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.457859 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.457888 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 15:08:56.458110 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 15:08:56.458237 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1098,"mutex_wait_us":94,"spinlock_wait_cycles":2816,"thread_start_us":87,"threads_started":1}
I20260811 15:08:56.460075 11521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.451175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:44543 (local address 127.0.0.1:42314)
0811 15:08:56.452848 (+ 1673us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:56.452862 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:56.452964 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:56.453438 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:56.453442 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:56.453467 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:56.453686 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.453697 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.455147 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.455151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.457426 (+ 2275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.457437 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.458866 (+ 1429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.458870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.459025 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.459055 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 15:08:56.459137 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 15:08:56.459219 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1608,"mutex_wait_us":1097,"thread_start_us":137,"threads_started":1}
I20260811 15:08:56.462183 11394 tablet_service.cc:1467] Processing CreateTablet for tablet 39dfe1673f4d45ecaadbd33f89d37859 (DEFAULT_TABLE table=test-table [id=429c2890cd3943b68d81a411ce726c97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:56.463660 11394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39dfe1673f4d45ecaadbd33f89d37859. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:56.463557 11178 tablet_service.cc:1467] Processing CreateTablet for tablet 39dfe1673f4d45ecaadbd33f89d37859 (DEFAULT_TABLE table=test-table [id=429c2890cd3943b68d81a411ce726c97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:56.463557 10968 tablet_service.cc:1467] Processing CreateTablet for tablet 39dfe1673f4d45ecaadbd33f89d37859 (DEFAULT_TABLE table=test-table [id=429c2890cd3943b68d81a411ce726c97]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:56.464922 10968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39dfe1673f4d45ecaadbd33f89d37859. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:56.464922 11178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 39dfe1673f4d45ecaadbd33f89d37859. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:56.472831 11532 tablet_bootstrap.cc:492] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f: Bootstrap starting.
I20260811 15:08:56.474620 11535 tablet_bootstrap.cc:492] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838: Bootstrap starting.
I20260811 15:08:56.475261 11534 tablet_bootstrap.cc:492] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce: Bootstrap starting.
I20260811 15:08:56.476563 11532 tablet_bootstrap.cc:654] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:56.477237 11535 tablet_bootstrap.cc:654] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:56.477371 11534 tablet_bootstrap.cc:654] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:56.477555 11532 log.cc:824] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:56.478215 11534 log.cc:824] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:56.478272 11535 log.cc:824] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838: Log is configured to *not* fsync() on all Append() calls
I20260811 15:08:56.480389 11532 tablet_bootstrap.cc:492] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f: No bootstrap required, opened a new log
I20260811 15:08:56.480638 11532 ts_tablet_manager.cc:1397] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 15:08:56.480664 11534 tablet_bootstrap.cc:492] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce: No bootstrap required, opened a new log
I20260811 15:08:56.480727 11535 tablet_bootstrap.cc:492] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838: No bootstrap required, opened a new log
I20260811 15:08:56.480913 11534 ts_tablet_manager.cc:1397] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 15:08:56.480939 11535 ts_tablet_manager.cc:1397] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 15:08:56.485096 11532 raft_consensus.cc:348] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } } peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } }
I20260811 15:08:56.485096 11535 raft_consensus.cc:348] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } } peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } }
I20260811 15:08:56.485441 11535 raft_consensus.cc:374] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:56.485441 11532 raft_consensus.cc:374] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:56.485496 11532 raft_consensus.cc:729] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79314a19dd6643c286c38475262d781f, State: Initialized, Role: FOLLOWER
I20260811 15:08:56.485496 11535 raft_consensus.cc:729] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982a70d4b9d44b51925104fde0947838, State: Initialized, Role: FOLLOWER
I20260811 15:08:56.485626 11534 raft_consensus.cc:348] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } } peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } }
I20260811 15:08:56.485962 11534 raft_consensus.cc:374] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:56.486039 11534 raft_consensus.cc:729] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88cf1c4770f64b97aef9328095fa61ce, State: Initialized, Role: FOLLOWER
I20260811 15:08:56.486104 11532 consensus_queue.cc:260] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } } peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } }
I20260811 15:08:56.486104 11535 consensus_queue.cc:260] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } } peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } }
I20260811 15:08:56.486636 11534 consensus_queue.cc:260] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } } peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } }
I20260811 15:08:56.487385 11532 ts_tablet_manager.cc:1428] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 15:08:56.487385 11535 ts_tablet_manager.cc:1428] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260811 15:08:56.487566 11484 heartbeater.cc:503] Master 127.30.8.62:44939 was elected leader, sending a full tablet report...
I20260811 15:08:56.487558 11287 heartbeater.cc:503] Master 127.30.8.62:44939 was elected leader, sending a full tablet report...
I20260811 15:08:56.488459 11062 heartbeater.cc:503] Master 127.30.8.62:44939 was elected leader, sending a full tablet report...
I20260811 15:08:56.488778 11534 ts_tablet_manager.cc:1428] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260811 15:08:56.560549 11542 raft_consensus.cc:482] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:08:56.560875 11542 raft_consensus.cc:504] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } } peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } }
I20260811 15:08:56.562530 11542 leader_election.cc:288] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 982a70d4b9d44b51925104fde0947838 (127.30.8.2:35591), 79314a19dd6643c286c38475262d781f (127.30.8.3:44281)
I20260811 15:08:56.567209 11529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.563092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:36561 (local address 127.30.8.2:35591)
0811 15:08:56.563254 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:56.563260 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:56.563522 (+ 262us) server_negotiation.cc:407] Connection header received
0811 15:08:56.563691 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:56.563694 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:56.563753 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:56.563838 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:56.564223 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.564824 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.566132 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.566564 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.566677 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.566758 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:56.567008 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 15:08:56.567094 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 15:08:56.567200 11544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.563084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:44281 (local address 127.30.8.1:51081)
0811 15:08:56.563543 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:56.563560 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:56.563710 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:56.564010 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:56.564012 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:56.564026 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:56.564180 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.564187 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.564977 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.564980 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.565953 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.565963 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.566656 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.566660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.566762 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.566789 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 15:08:56.566867 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 15:08:56.566985 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":47,"thread_start_us":166,"threads_started":1}
I20260811 15:08:56.567210 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.563136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:51081 (local address 127.30.8.3:44281)
0811 15:08:56.563398 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:56.563402 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:56.563698 (+ 296us) server_negotiation.cc:407] Connection header received
0811 15:08:56.563829 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:56.563832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:56.563896 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:56.563984 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:56.564308 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.564854 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.566102 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.566522 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.566610 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.566692 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:56.566955 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 15:08:56.567053 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145}
I20260811 15:08:56.567262 11543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.562875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:35591 (local address 127.30.8.1:36561)
0811 15:08:56.563396 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:56.563414 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:56.563566 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:56.563872 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:56.563876 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:56.563895 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:56.564098 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.564108 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.564960 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.564964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.565986 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.565996 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.566713 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.566716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.566817 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.566844 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 15:08:56.566984 (+ 140us) client_negotiation.cc:240] Negotiation successful
0811 15:08:56.567080 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":427,"mutex_wait_us":56,"thread_start_us":197,"threads_started":1}
I20260811 15:08:56.568163 11426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39dfe1673f4d45ecaadbd33f89d37859" candidate_uuid: "88cf1c4770f64b97aef9328095fa61ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79314a19dd6643c286c38475262d781f" is_pre_election: true
I20260811 15:08:56.568264 11207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39dfe1673f4d45ecaadbd33f89d37859" candidate_uuid: "88cf1c4770f64b97aef9328095fa61ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982a70d4b9d44b51925104fde0947838" is_pre_election: true
I20260811 15:08:56.568531 11426 raft_consensus.cc:2454] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88cf1c4770f64b97aef9328095fa61ce in term 0.
I20260811 15:08:56.568579 11207 raft_consensus.cc:2454] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88cf1c4770f64b97aef9328095fa61ce in term 0.
I20260811 15:08:56.569092 10890 leader_election.cc:302] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79314a19dd6643c286c38475262d781f, 88cf1c4770f64b97aef9328095fa61ce; no voters:
I20260811 15:08:56.569434 11542 raft_consensus.cc:2781] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:08:56.569523 11542 raft_consensus.cc:482] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:08:56.569554 11542 raft_consensus.cc:3037] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:56.570717 11542 raft_consensus.cc:504] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } } peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } }
I20260811 15:08:56.571141 11542 leader_election.cc:288] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [CANDIDATE]: Term 1 election: Requested vote from peers 982a70d4b9d44b51925104fde0947838 (127.30.8.2:35591), 79314a19dd6643c286c38475262d781f (127.30.8.3:44281)
I20260811 15:08:56.571499 11426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39dfe1673f4d45ecaadbd33f89d37859" candidate_uuid: "88cf1c4770f64b97aef9328095fa61ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79314a19dd6643c286c38475262d781f"
I20260811 15:08:56.571533 11207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "39dfe1673f4d45ecaadbd33f89d37859" candidate_uuid: "88cf1c4770f64b97aef9328095fa61ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982a70d4b9d44b51925104fde0947838"
I20260811 15:08:56.571616 11426 raft_consensus.cc:3037] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:56.571661 11207 raft_consensus.cc:3037] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:56.572775 11426 raft_consensus.cc:2454] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88cf1c4770f64b97aef9328095fa61ce in term 1.
I20260811 15:08:56.572799 11207 raft_consensus.cc:2454] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88cf1c4770f64b97aef9328095fa61ce in term 1.
I20260811 15:08:56.573089 10890 leader_election.cc:302] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 79314a19dd6643c286c38475262d781f, 88cf1c4770f64b97aef9328095fa61ce, 982a70d4b9d44b51925104fde0947838; no voters:
I20260811 15:08:56.573258 11542 raft_consensus.cc:2781] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:56.573505 11542 raft_consensus.cc:686] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [term 1 LEADER]: Becoming Leader. State: Replica: 88cf1c4770f64b97aef9328095fa61ce, State: Running, Role: LEADER
I20260811 15:08:56.573869 11542 consensus_queue.cc:237] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } } peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } }
I20260811 15:08:56.577406 10757 catalog_manager.cc:5495] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 88cf1c4770f64b97aef9328095fa61ce (127.30.8.1). New cstate: current_term: 1 leader_uuid: "88cf1c4770f64b97aef9328095fa61ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } health_report { overall_health: HEALTHY } } }
I20260811 15:08:56.614759 11530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.610623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42320 (local address 127.30.8.1:44543)
0811 15:08:56.610760 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:56.610764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:56.610776 (+ 12us) server_negotiation.cc:407] Connection header received
0811 15:08:56.610858 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:56.610861 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:56.610950 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:56.611056 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:56.611308 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.611815 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.612726 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.613030 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.613172 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.613478 (+ 306us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:56.614153 (+ 675us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:56.614266 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:56.614483 (+ 217us) server_negotiation.cc:299] Negotiation successful
0811 15:08:56.614586 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 15:08:56.622304 11207 raft_consensus.cc:1261] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838 [term 1 FOLLOWER]: Refusing update from remote peer 88cf1c4770f64b97aef9328095fa61ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:56.622551 11426 raft_consensus.cc:1261] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f [term 1 FOLLOWER]: Refusing update from remote peer 88cf1c4770f64b97aef9328095fa61ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:56.623266 11545 consensus_queue.cc:1035] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:56.623472 11542 consensus_queue.cc:1035] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 15:08:56.629400 11064 tablet.cc:2369] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:08:56.633375 11550 mvcc.cc:205] Tried to move back new op lower bound from 7317343996395241472 to 7317343996209397760. Current Snapshot: MvccSnapshot[applied={T|T < 7317343996395241472}]
I20260811 15:08:56.635565 11554 mvcc.cc:205] Tried to move back new op lower bound from 7317343996395241472 to 7317343996209397760. Current Snapshot: MvccSnapshot[applied={T|T < 7317343996395241472}]
I20260811 15:08:56.642203 10757 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:56.642387 10757 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:56.644906 10757 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49: Sending DeleteTablet for 3 replicas of tablet 39dfe1673f4d45ecaadbd33f89d37859
I20260811 15:08:56.645892 11394 tablet_service.cc:1514] Processing DeleteTablet for tablet 39dfe1673f4d45ecaadbd33f89d37859 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:56 UTC) from {username='oryx', principal='oryx/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:53276
I20260811 15:08:56.645876 11178 tablet_service.cc:1514] Processing DeleteTablet for tablet 39dfe1673f4d45ecaadbd33f89d37859 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:56 UTC) from {username='oryx', principal='oryx/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:34838
I20260811 15:08:56.646499 11569 tablet_replica.cc:330] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f: stopping tablet replica
I20260811 15:08:56.646512 10968 tablet_service.cc:1514] Processing DeleteTablet for tablet 39dfe1673f4d45ecaadbd33f89d37859 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:56 UTC) from {username='oryx', principal='oryx/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:42314
I20260811 15:08:56.646654 11568 tablet_replica.cc:330] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838: stopping tablet replica
I20260811 15:08:56.646894 11569 raft_consensus.cc:2229] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:56.647058 11568 raft_consensus.cc:2229] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:56.647292 11569 raft_consensus.cc:2258] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:56.647411 11568 raft_consensus.cc:2258] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:56.648027 10757 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54298:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:08:56.648512 10757 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:08:56.649399 11569 ts_tablet_manager.cc:1905] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:56.650238 11568 ts_tablet_manager.cc:1905] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:56.652802 11569 ts_tablet_manager.cc:1918] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:08:56.652921 11569 log.cc:1196] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/39dfe1673f4d45ecaadbd33f89d37859
I20260811 15:08:56.653292 11569 ts_tablet_manager.cc:1939] T 39dfe1673f4d45ecaadbd33f89d37859 P 79314a19dd6643c286c38475262d781f: Deleting consensus metadata
I20260811 15:08:56.654358 10968 tablet_service.cc:1467] Processing CreateTablet for tablet 18f81a0453ad40479afec6c162b4fefb (DEFAULT_TABLE table=test-table [id=65b5190a126e4831ac3c76baf6edcd6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:56.654655 10968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18f81a0453ad40479afec6c162b4fefb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:56.654845 11570 tablet_replica.cc:330] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce: stopping tablet replica
I20260811 15:08:56.655243 11570 raft_consensus.cc:2229] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:08:56.655207 11394 tablet_service.cc:1467] Processing CreateTablet for tablet 18f81a0453ad40479afec6c162b4fefb (DEFAULT_TABLE table=test-table [id=65b5190a126e4831ac3c76baf6edcd6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:56.655471 11394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18f81a0453ad40479afec6c162b4fefb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:56.655722 11570 raft_consensus.cc:2258] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:56.657245 11534 tablet_bootstrap.cc:492] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce: Bootstrap starting.
I20260811 15:08:56.657801 10734 catalog_manager.cc:4841] TS 79314a19dd6643c286c38475262d781f (127.30.8.3:44281): tablet 39dfe1673f4d45ecaadbd33f89d37859 (table test-table [id=429c2890cd3943b68d81a411ce726c97]) successfully deleted
I20260811 15:08:56.658231 11534 tablet_bootstrap.cc:654] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:56.658228 11178 tablet_service.cc:1467] Processing CreateTablet for tablet 18f81a0453ad40479afec6c162b4fefb (DEFAULT_TABLE table=test-table [id=65b5190a126e4831ac3c76baf6edcd6c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:56.658819 11178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 18f81a0453ad40479afec6c162b4fefb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:56.658893 11532 tablet_bootstrap.cc:492] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f: Bootstrap starting.
I20260811 15:08:56.659895 11532 tablet_bootstrap.cc:654] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:56.660187 11568 ts_tablet_manager.cc:1918] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:08:56.660291 11568 log.cc:1196] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/39dfe1673f4d45ecaadbd33f89d37859
I20260811 15:08:56.660626 11568 ts_tablet_manager.cc:1939] T 39dfe1673f4d45ecaadbd33f89d37859 P 982a70d4b9d44b51925104fde0947838: Deleting consensus metadata
I20260811 15:08:56.661470 11535 tablet_bootstrap.cc:492] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838: Bootstrap starting.
I20260811 15:08:56.662083 10731 catalog_manager.cc:4841] TS 982a70d4b9d44b51925104fde0947838 (127.30.8.2:35591): tablet 39dfe1673f4d45ecaadbd33f89d37859 (table test-table [id=429c2890cd3943b68d81a411ce726c97]) successfully deleted
I20260811 15:08:56.662447 11535 tablet_bootstrap.cc:654] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:56.663166 11570 ts_tablet_manager.cc:1905] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:56.664316 11532 tablet_bootstrap.cc:492] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f: No bootstrap required, opened a new log
I20260811 15:08:56.664426 11532 ts_tablet_manager.cc:1397] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.002s
I20260811 15:08:56.664533 11534 tablet_bootstrap.cc:492] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce: No bootstrap required, opened a new log
I20260811 15:08:56.664624 11534 ts_tablet_manager.cc:1397] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 15:08:56.664989 11532 raft_consensus.cc:348] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:56.665120 11532 raft_consensus.cc:374] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:56.665058 11534 raft_consensus.cc:348] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:56.665158 11532 raft_consensus.cc:729] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79314a19dd6643c286c38475262d781f, State: Initialized, Role: FOLLOWER
I20260811 15:08:56.665163 11534 raft_consensus.cc:374] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:56.665194 11534 raft_consensus.cc:729] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88cf1c4770f64b97aef9328095fa61ce, State: Initialized, Role: FOLLOWER
I20260811 15:08:56.665311 11534 consensus_queue.cc:260] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:56.665305 11532 consensus_queue.cc:260] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:56.665556 11534 ts_tablet_manager.cc:1428] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:56.665647 11532 ts_tablet_manager.cc:1428] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 15:08:56.667032 11570 ts_tablet_manager.cc:1918] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:08:56.667146 11570 log.cc:1196] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-0/wal/wals/39dfe1673f4d45ecaadbd33f89d37859
I20260811 15:08:56.667465 11570 ts_tablet_manager.cc:1939] T 39dfe1673f4d45ecaadbd33f89d37859 P 88cf1c4770f64b97aef9328095fa61ce: Deleting consensus metadata
I20260811 15:08:56.668690 10736 catalog_manager.cc:4841] TS 88cf1c4770f64b97aef9328095fa61ce (127.30.8.1:44543): tablet 39dfe1673f4d45ecaadbd33f89d37859 (table test-table [id=429c2890cd3943b68d81a411ce726c97]) successfully deleted
I20260811 15:08:56.669555 11535 tablet_bootstrap.cc:492] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838: No bootstrap required, opened a new log
I20260811 15:08:56.669675 11535 ts_tablet_manager.cc:1397] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.001s
I20260811 15:08:56.670244 11535 raft_consensus.cc:348] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:56.670384 11535 raft_consensus.cc:374] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:56.670425 11535 raft_consensus.cc:729] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982a70d4b9d44b51925104fde0947838, State: Initialized, Role: FOLLOWER
I20260811 15:08:56.670593 11535 consensus_queue.cc:260] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:56.670854 11535 ts_tablet_manager.cc:1428] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260811 15:08:56.875519 11561 raft_consensus.cc:482] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:08:56.875669 11561 raft_consensus.cc:504] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:56.876134 11561 leader_election.cc:288] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 982a70d4b9d44b51925104fde0947838 (127.30.8.2:35591), 79314a19dd6643c286c38475262d781f (127.30.8.3:44281)
I20260811 15:08:56.876699 11207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18f81a0453ad40479afec6c162b4fefb" candidate_uuid: "88cf1c4770f64b97aef9328095fa61ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982a70d4b9d44b51925104fde0947838" is_pre_election: true
I20260811 15:08:56.876857 11207 raft_consensus.cc:2454] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88cf1c4770f64b97aef9328095fa61ce in term 0.
I20260811 15:08:56.876838 11426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18f81a0453ad40479afec6c162b4fefb" candidate_uuid: "88cf1c4770f64b97aef9328095fa61ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79314a19dd6643c286c38475262d781f" is_pre_election: true
I20260811 15:08:56.877010 11426 raft_consensus.cc:2454] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88cf1c4770f64b97aef9328095fa61ce in term 0.
I20260811 15:08:56.877198 10888 leader_election.cc:302] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88cf1c4770f64b97aef9328095fa61ce, 982a70d4b9d44b51925104fde0947838; no voters:
I20260811 15:08:56.877419 11561 raft_consensus.cc:2781] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:08:56.877478 11561 raft_consensus.cc:482] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:08:56.877517 11561 raft_consensus.cc:3037] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:56.878513 11561 raft_consensus.cc:504] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:56.878836 11561 leader_election.cc:288] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [CANDIDATE]: Term 1 election: Requested vote from peers 982a70d4b9d44b51925104fde0947838 (127.30.8.2:35591), 79314a19dd6643c286c38475262d781f (127.30.8.3:44281)
I20260811 15:08:56.879194 11207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18f81a0453ad40479afec6c162b4fefb" candidate_uuid: "88cf1c4770f64b97aef9328095fa61ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982a70d4b9d44b51925104fde0947838"
I20260811 15:08:56.879303 11207 raft_consensus.cc:3037] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:56.880312 11207 raft_consensus.cc:2454] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88cf1c4770f64b97aef9328095fa61ce in term 1.
I20260811 15:08:56.880586 10888 leader_election.cc:302] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88cf1c4770f64b97aef9328095fa61ce, 982a70d4b9d44b51925104fde0947838; no voters:
I20260811 15:08:56.880556 11426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "18f81a0453ad40479afec6c162b4fefb" candidate_uuid: "88cf1c4770f64b97aef9328095fa61ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79314a19dd6643c286c38475262d781f"
I20260811 15:08:56.880707 11426 raft_consensus.cc:3037] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:56.880756 11561 raft_consensus.cc:2781] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:56.880837 11561 raft_consensus.cc:686] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [term 1 LEADER]: Becoming Leader. State: Replica: 88cf1c4770f64b97aef9328095fa61ce, State: Running, Role: LEADER
I20260811 15:08:56.880973 11561 consensus_queue.cc:237] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:56.881819 11426 raft_consensus.cc:2454] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88cf1c4770f64b97aef9328095fa61ce in term 1.
I20260811 15:08:56.882772 10755 catalog_manager.cc:5495] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 88cf1c4770f64b97aef9328095fa61ce (127.30.8.1). New cstate: current_term: 1 leader_uuid: "88cf1c4770f64b97aef9328095fa61ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } health_report { overall_health: UNKNOWN } } }
W20260811 15:08:56.885748 11485 tablet.cc:2369] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 15:08:56.886155 11288 tablet.cc:2369] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 15:08:56 dist-test-slave-1vmg krb5kdc[10682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.62@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM
I20260811 15:08:56.924355 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.916819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54312 (local address 127.30.8.62:44939)
0811 15:08:56.916975 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:56.916979 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:56.917099 (+ 120us) server_negotiation.cc:407] Connection header received
0811 15:08:56.917222 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:56.917225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:56.917267 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:56.917334 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:56.917928 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.918436 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.919226 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.919430 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.919509 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.921704 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:56.921708 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:56.921710 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:56.921733 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:56.922820 (+ 1087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:56.923272 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:56.923275 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:56.923276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:56.923347 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:56.923540 (+ 193us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:56.923542 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:56.923543 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:56.923709 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:56.923779 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:56.924016 (+ 237us) server_negotiation.cc:299] Negotiation successful
0811 15:08:56.924199 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":82,"server-negotiator.queue_time_us":70}
I20260811 15:08:56.924357 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.916754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:44939 (local address 127.0.0.1:54312)
0811 15:08:56.917019 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:56.917031 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:56.917148 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:56.917358 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:56.917360 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:56.917744 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:08:56.917840 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.917846 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.918513 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.918515 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.919115 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.919124 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.919515 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.919517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.919574 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.920131 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:08:56.920143 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:08:56.921576 (+ 1433us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:08:56.922909 (+ 1333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:56.922951 (+ 42us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:56.922959 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:56.923194 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:56.923411 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:08:56.923413 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:08:56.923414 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:08:56.923484 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:08:56.923782 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:08:56.923788 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:08:56.923891 (+ 103us) client_negotiation.cc:769] Sending connection context
0811 15:08:56.923977 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 15:08:56.924150 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":111,"threads_started":1}
I20260811 15:08:56.928604 10755 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:54312:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 15:08:56.929325 10755 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:08:56.933868 10968 tablet_service.cc:1467] Processing CreateTablet for tablet 0f65bddb55094a3a92d16d22dfcf1837 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7810306667d944df8b1834bc97918e6f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 15:08:56.933938 11178 tablet_service.cc:1467] Processing CreateTablet for tablet 0f65bddb55094a3a92d16d22dfcf1837 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7810306667d944df8b1834bc97918e6f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 15:08:56.933955 11394 tablet_service.cc:1467] Processing CreateTablet for tablet 0f65bddb55094a3a92d16d22dfcf1837 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7810306667d944df8b1834bc97918e6f]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 15:08:56.934306 11394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f65bddb55094a3a92d16d22dfcf1837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:56.934306 10968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f65bddb55094a3a92d16d22dfcf1837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:56.934319 11178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f65bddb55094a3a92d16d22dfcf1837. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:56.936915 11535 tablet_bootstrap.cc:492] T 0f65bddb55094a3a92d16d22dfcf1837 P 982a70d4b9d44b51925104fde0947838: Bootstrap starting.
I20260811 15:08:56.937043 11534 tablet_bootstrap.cc:492] T 0f65bddb55094a3a92d16d22dfcf1837 P 88cf1c4770f64b97aef9328095fa61ce: Bootstrap starting.
I20260811 15:08:56.937772 11534 tablet_bootstrap.cc:654] T 0f65bddb55094a3a92d16d22dfcf1837 P 88cf1c4770f64b97aef9328095fa61ce: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:56.937785 11535 tablet_bootstrap.cc:654] T 0f65bddb55094a3a92d16d22dfcf1837 P 982a70d4b9d44b51925104fde0947838: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:56.938508 11532 tablet_bootstrap.cc:492] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f: Bootstrap starting.
I20260811 15:08:56.938884 11535 tablet_bootstrap.cc:492] T 0f65bddb55094a3a92d16d22dfcf1837 P 982a70d4b9d44b51925104fde0947838: No bootstrap required, opened a new log
I20260811 15:08:56.939009 11535 ts_tablet_manager.cc:1397] T 0f65bddb55094a3a92d16d22dfcf1837 P 982a70d4b9d44b51925104fde0947838: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260811 15:08:56.939368 11532 tablet_bootstrap.cc:654] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:56.939360 11534 tablet_bootstrap.cc:492] T 0f65bddb55094a3a92d16d22dfcf1837 P 88cf1c4770f64b97aef9328095fa61ce: No bootstrap required, opened a new log
I20260811 15:08:56.939446 11534 ts_tablet_manager.cc:1397] T 0f65bddb55094a3a92d16d22dfcf1837 P 88cf1c4770f64b97aef9328095fa61ce: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 15:08:56.939466 11535 raft_consensus.cc:348] T 0f65bddb55094a3a92d16d22dfcf1837 P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:56.939585 11535 raft_consensus.cc:374] T 0f65bddb55094a3a92d16d22dfcf1837 P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:56.939625 11535 raft_consensus.cc:729] T 0f65bddb55094a3a92d16d22dfcf1837 P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982a70d4b9d44b51925104fde0947838, State: Initialized, Role: FOLLOWER
I20260811 15:08:56.939771 11535 consensus_queue.cc:260] T 0f65bddb55094a3a92d16d22dfcf1837 P 982a70d4b9d44b51925104fde0947838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:56.939885 11534 raft_consensus.cc:348] T 0f65bddb55094a3a92d16d22dfcf1837 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:56.939988 11534 raft_consensus.cc:374] T 0f65bddb55094a3a92d16d22dfcf1837 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:56.940026 11534 raft_consensus.cc:729] T 0f65bddb55094a3a92d16d22dfcf1837 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88cf1c4770f64b97aef9328095fa61ce, State: Initialized, Role: FOLLOWER
I20260811 15:08:56.940060 11535 ts_tablet_manager.cc:1428] T 0f65bddb55094a3a92d16d22dfcf1837 P 982a70d4b9d44b51925104fde0947838: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 15:08:56.940151 11534 consensus_queue.cc:260] T 0f65bddb55094a3a92d16d22dfcf1837 P 88cf1c4770f64b97aef9328095fa61ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:56.940272 11541 tablet_replica.cc:439] T 0f65bddb55094a3a92d16d22dfcf1837 P 982a70d4b9d44b51925104fde0947838: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } } }
I20260811 15:08:56.940402 11541 tablet_replica.cc:442] T 0f65bddb55094a3a92d16d22dfcf1837 P 982a70d4b9d44b51925104fde0947838: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:56.940402 11534 ts_tablet_manager.cc:1428] T 0f65bddb55094a3a92d16d22dfcf1837 P 88cf1c4770f64b97aef9328095fa61ce: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 15:08:56.940510 11532 tablet_bootstrap.cc:492] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f: No bootstrap required, opened a new log
I20260811 15:08:56.940588 11532 ts_tablet_manager.cc:1397] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 15:08:56.940575 11561 tablet_replica.cc:439] T 0f65bddb55094a3a92d16d22dfcf1837 P 88cf1c4770f64b97aef9328095fa61ce: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } } }
I20260811 15:08:56.940686 11561 tablet_replica.cc:442] T 0f65bddb55094a3a92d16d22dfcf1837 P 88cf1c4770f64b97aef9328095fa61ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:56.941028 11532 raft_consensus.cc:348] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:56.941123 11532 raft_consensus.cc:374] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:56.941154 11532 raft_consensus.cc:729] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79314a19dd6643c286c38475262d781f, State: Initialized, Role: FOLLOWER
I20260811 15:08:56.941285 11532 consensus_queue.cc:260] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:56.941519 11540 tablet_replica.cc:439] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } } }
I20260811 15:08:56.941672 11540 tablet_replica.cc:442] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:56.941552 11532 ts_tablet_manager.cc:1428] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 15:08:56.947691 11540 raft_consensus.cc:482] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:08:56.947793 11540 raft_consensus.cc:504] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:56.948796 11540 leader_election.cc:288] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88cf1c4770f64b97aef9328095fa61ce (127.30.8.1:44543), 982a70d4b9d44b51925104fde0947838 (127.30.8.2:35591)
I20260811 15:08:56.952941 11530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.949335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:36239 (local address 127.30.8.1:44543)
0811 15:08:56.949463 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:56.949467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:56.949478 (+ 11us) server_negotiation.cc:407] Connection header received
0811 15:08:56.949521 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:56.949524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:56.949566 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:56.949640 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:56.949957 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.950756 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.951902 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.952346 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.952452 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.952522 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:56.952748 (+ 226us) server_negotiation.cc:299] Negotiation successful
0811 15:08:56.952818 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 15:08:56.952941 11529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.949133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:56349 (local address 127.30.8.2:35591)
0811 15:08:56.949280 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:56.949284 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:56.949605 (+ 321us) server_negotiation.cc:407] Connection header received
0811 15:08:56.949718 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:56.949721 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:56.949767 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:56.949835 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:56.950108 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.950852 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.951952 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.952372 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.952463 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.952529 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:56.952738 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 15:08:56.952812 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 15:08:56.952970 11590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.949032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:35591 (local address 127.30.8.3:56349)
0811 15:08:56.949540 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:56.949552 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:56.949635 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:56.949860 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:56.949863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:56.949879 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:56.949985 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.949992 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.951005 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.951007 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.951837 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.951847 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.952466 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.952470 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.952604 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.952645 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 15:08:56.952708 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 15:08:56.952809 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":36,"thread_start_us":114,"threads_started":1}
I20260811 15:08:56.952973 11589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.948986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:44543 (local address 127.30.8.3:36239)
0811 15:08:56.949258 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:56.949273 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:56.949364 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:56.949666 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:56.949670 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:56.949685 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:56.949818 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.949826 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.950849 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.950852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.951786 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.951798 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.952467 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.952470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.952631 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.952653 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 15:08:56.952717 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 15:08:56.952822 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"mutex_wait_us":49,"thread_start_us":97,"threads_started":1}
I20260811 15:08:56.953538 11207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f65bddb55094a3a92d16d22dfcf1837" candidate_uuid: "79314a19dd6643c286c38475262d781f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982a70d4b9d44b51925104fde0947838" is_pre_election: true
I20260811 15:08:56.953611 10993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f65bddb55094a3a92d16d22dfcf1837" candidate_uuid: "79314a19dd6643c286c38475262d781f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88cf1c4770f64b97aef9328095fa61ce" is_pre_election: true
I20260811 15:08:56.953678 11207 raft_consensus.cc:2454] T 0f65bddb55094a3a92d16d22dfcf1837 P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79314a19dd6643c286c38475262d781f in term 0.
I20260811 15:08:56.953840 10993 raft_consensus.cc:2454] T 0f65bddb55094a3a92d16d22dfcf1837 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79314a19dd6643c286c38475262d781f in term 0.
I20260811 15:08:56.954034 11321 leader_election.cc:302] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79314a19dd6643c286c38475262d781f, 982a70d4b9d44b51925104fde0947838; no voters:
I20260811 15:08:56.954300 11540 raft_consensus.cc:2781] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:08:56.954361 11540 raft_consensus.cc:482] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:08:56.954407 11540 raft_consensus.cc:3037] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:56.955539 11540 raft_consensus.cc:504] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:56.955847 11540 leader_election.cc:288] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f [CANDIDATE]: Term 1 election: Requested vote from peers 88cf1c4770f64b97aef9328095fa61ce (127.30.8.1:44543), 982a70d4b9d44b51925104fde0947838 (127.30.8.2:35591)
I20260811 15:08:56.956169 10993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f65bddb55094a3a92d16d22dfcf1837" candidate_uuid: "79314a19dd6643c286c38475262d781f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88cf1c4770f64b97aef9328095fa61ce"
I20260811 15:08:56.956202 11207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f65bddb55094a3a92d16d22dfcf1837" candidate_uuid: "79314a19dd6643c286c38475262d781f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982a70d4b9d44b51925104fde0947838"
I20260811 15:08:56.956295 11207 raft_consensus.cc:3037] T 0f65bddb55094a3a92d16d22dfcf1837 P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:56.956293 10993 raft_consensus.cc:3037] T 0f65bddb55094a3a92d16d22dfcf1837 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:56.957347 11207 raft_consensus.cc:2454] T 0f65bddb55094a3a92d16d22dfcf1837 P 982a70d4b9d44b51925104fde0947838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79314a19dd6643c286c38475262d781f in term 1.
I20260811 15:08:56.957347 10993 raft_consensus.cc:2454] T 0f65bddb55094a3a92d16d22dfcf1837 P 88cf1c4770f64b97aef9328095fa61ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79314a19dd6643c286c38475262d781f in term 1.
I20260811 15:08:56.957631 11327 leader_election.cc:302] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79314a19dd6643c286c38475262d781f, 88cf1c4770f64b97aef9328095fa61ce; no voters:
I20260811 15:08:56.957798 11540 raft_consensus.cc:2781] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:56.957991 11540 raft_consensus.cc:686] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f [term 1 LEADER]: Becoming Leader. State: Replica: 79314a19dd6643c286c38475262d781f, State: Running, Role: LEADER
I20260811 15:08:56.958240 11540 consensus_queue.cc:237] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:56.959506 11593 tablet_replica.cc:439] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f: TxnStatusTablet state changed. Reason: New leader 79314a19dd6643c286c38475262d781f. Latest consensus state: current_term: 1 leader_uuid: "79314a19dd6643c286c38475262d781f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } } }
I20260811 15:08:56.959616 11593 tablet_replica.cc:442] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f: This TxnStatusTablet replica's current role is: LEADER
I20260811 15:08:56.960094 11596 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 15:08:56.960239 11596 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 15:08:56.960713 10757 catalog_manager.cc:5495] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f reported cstate change: term changed from 0 to 1, leader changed from <none> to 79314a19dd6643c286c38475262d781f (127.30.8.3). New cstate: current_term: 1 leader_uuid: "79314a19dd6643c286c38475262d781f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260811 15:08:56.976856 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.973360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53290 (local address 127.30.8.3:44281)
0811 15:08:56.973497 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:56.973502 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:56.973517 (+ 15us) server_negotiation.cc:407] Connection header received
0811 15:08:56.973589 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:56.973592 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:56.973667 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:56.973754 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:56.973987 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.974509 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.975399 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.975619 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.975705 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.976002 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:56.976460 (+ 458us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:56.976565 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:56.976647 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 15:08:56.976703 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 15:08:56.976856 11584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.973195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:44281 (local address 127.0.0.1:53290)
0811 15:08:56.973347 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:56.973363 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:56.973465 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:56.973773 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:56.973776 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:56.973786 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 15:08:56.973890 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.973897 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.974612 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.974616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.975287 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.975297 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.975714 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.975718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.975800 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.975846 (+ 46us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 15:08:56.976557 (+ 711us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 15:08:56.976572 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 15:08:56.976637 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 15:08:56.976728 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 15:08:56.980360 11207 raft_consensus.cc:1261] T 0f65bddb55094a3a92d16d22dfcf1837 P 982a70d4b9d44b51925104fde0947838 [term 1 FOLLOWER]: Refusing update from remote peer 79314a19dd6643c286c38475262d781f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:56.980541 10993 raft_consensus.cc:1261] T 0f65bddb55094a3a92d16d22dfcf1837 P 88cf1c4770f64b97aef9328095fa61ce [term 1 FOLLOWER]: Refusing update from remote peer 79314a19dd6643c286c38475262d781f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:56.981038 11540 consensus_queue.cc:1035] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f [LEADER]: Connected to new peer: Peer: permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:56.981447 11593 consensus_queue.cc:1035] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f [LEADER]: Connected to new peer: Peer: permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:56.982616 11541 tablet_replica.cc:439] T 0f65bddb55094a3a92d16d22dfcf1837 P 982a70d4b9d44b51925104fde0947838: TxnStatusTablet state changed. Reason: New leader 79314a19dd6643c286c38475262d781f. Latest consensus state: current_term: 1 leader_uuid: "79314a19dd6643c286c38475262d781f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } } }
I20260811 15:08:56.982733 11541 tablet_replica.cc:442] T 0f65bddb55094a3a92d16d22dfcf1837 P 982a70d4b9d44b51925104fde0947838: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:56.983793 11561 tablet_replica.cc:439] T 0f65bddb55094a3a92d16d22dfcf1837 P 88cf1c4770f64b97aef9328095fa61ce: TxnStatusTablet state changed. Reason: New leader 79314a19dd6643c286c38475262d781f. Latest consensus state: current_term: 1 leader_uuid: "79314a19dd6643c286c38475262d781f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } } }
I20260811 15:08:56.983902 11561 tablet_replica.cc:442] T 0f65bddb55094a3a92d16d22dfcf1837 P 88cf1c4770f64b97aef9328095fa61ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:56.985702 11540 tablet_replica.cc:439] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "79314a19dd6643c286c38475262d781f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } } }
I20260811 15:08:56.985724 11593 tablet_replica.cc:439] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "79314a19dd6643c286c38475262d781f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } } }
I20260811 15:08:56.985812 11540 tablet_replica.cc:442] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f: This TxnStatusTablet replica's current role is: LEADER
I20260811 15:08:56.985824 11593 tablet_replica.cc:442] T 0f65bddb55094a3a92d16d22dfcf1837 P 79314a19dd6643c286c38475262d781f: This TxnStatusTablet replica's current role is: LEADER
I20260811 15:08:56.986889 11549 mvcc.cc:205] Tried to move back new op lower bound from 7317343997864497152 to 7317343997783420928. Current Snapshot: MvccSnapshot[applied={T|T < 7317343997864497152}]
I20260811 15:08:56.987093 11561 tablet_replica.cc:439] T 0f65bddb55094a3a92d16d22dfcf1837 P 88cf1c4770f64b97aef9328095fa61ce: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "79314a19dd6643c286c38475262d781f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } } }
I20260811 15:08:56.987094 11541 tablet_replica.cc:439] T 0f65bddb55094a3a92d16d22dfcf1837 P 982a70d4b9d44b51925104fde0947838: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "79314a19dd6643c286c38475262d781f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } } }
I20260811 15:08:56.987195 11561 tablet_replica.cc:442] T 0f65bddb55094a3a92d16d22dfcf1837 P 88cf1c4770f64b97aef9328095fa61ce: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:56.987202 11541 tablet_replica.cc:442] T 0f65bddb55094a3a92d16d22dfcf1837 P 982a70d4b9d44b51925104fde0947838: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 15:08:57.000504 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.997037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:40009 (local address 127.30.8.3:44281)
0811 15:08:56.997173 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:56.997177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:56.997281 (+ 104us) server_negotiation.cc:407] Connection header received
0811 15:08:56.997411 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:56.997413 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:56.997457 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:56.997516 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:56.997746 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.998233 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.999444 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.999655 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.999758 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:57.000035 (+ 277us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:57.000126 (+ 91us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:57.000184 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:57.000309 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 15:08:57.000361 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 15:08:57.000522 11605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:56.996900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:44281 (local address 127.30.8.1:40009)
0811 15:08:56.997201 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:56.997216 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:56.997323 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:56.997536 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:56.997539 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:56.997547 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 15:08:56.997656 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.997662 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.998319 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.998322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.999337 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:56.999347 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:56.999764 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:56.999767 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:56.999851 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:56.999908 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 15:08:57.000211 (+ 303us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 15:08:57.000226 (+ 15us) client_negotiation.cc:769] Sending connection context
0811 15:08:57.000291 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 15:08:57.000397 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":95,"threads_started":1}
I20260811 15:08:57.007632 11207 raft_consensus.cc:1261] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838 [term 1 FOLLOWER]: Refusing update from remote peer 88cf1c4770f64b97aef9328095fa61ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:57.007680 11426 raft_consensus.cc:1261] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f [term 1 FOLLOWER]: Refusing update from remote peer 88cf1c4770f64b97aef9328095fa61ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:57.008006 11545 consensus_queue.cc:1035] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:57.008157 11561 consensus_queue.cc:1035] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:57.032332 11530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:57.028024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:41899 (local address 127.30.8.1:44543)
0811 15:08:57.028139 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:57.028143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:57.028358 (+ 215us) server_negotiation.cc:407] Connection header received
0811 15:08:57.028494 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:57.028498 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:57.028552 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:57.028636 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:57.028902 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.029705 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.030988 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.031264 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.031362 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:57.031702 (+ 340us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:57.031789 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:57.031832 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:57.032125 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 15:08:57.032186 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 15:08:57.032739 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:57.027830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:44543 (local address 127.30.8.3:41899)
0811 15:08:57.028276 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:57.028292 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:57.028404 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:57.028646 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:57.028649 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:57.028655 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 15:08:57.028775 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:57.028781 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.029845 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.029848 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:57.030842 (+ 994us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:57.030853 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.031364 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.031368 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:57.031467 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:57.031539 (+ 72us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 15:08:57.032011 (+ 472us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 15:08:57.032018 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 15:08:57.032490 (+ 472us) client_negotiation.cc:240] Negotiation successful
0811 15:08:57.032588 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":116,"threads_started":1}
I20260811 15:08:57.055655 10754 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:57.055812 10754 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54298:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:57.057821 10754 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49: Sending DeleteTablet for 3 replicas of tablet 18f81a0453ad40479afec6c162b4fefb
I20260811 15:08:57.058339 10968 tablet_service.cc:1514] Processing DeleteTablet for tablet 18f81a0453ad40479afec6c162b4fefb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:57 UTC) from {username='oryx', principal='oryx/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:42314
I20260811 15:08:57.058480 11178 tablet_service.cc:1514] Processing DeleteTablet for tablet 18f81a0453ad40479afec6c162b4fefb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:57 UTC) from {username='oryx', principal='oryx/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:34838
I20260811 15:08:57.058471 11394 tablet_service.cc:1514] Processing DeleteTablet for tablet 18f81a0453ad40479afec6c162b4fefb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:57 UTC) from {username='oryx', principal='oryx/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:53276
I20260811 15:08:57.058549 11570 tablet_replica.cc:330] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce: stopping tablet replica
I20260811 15:08:57.058647 11568 tablet_replica.cc:330] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838: stopping tablet replica
I20260811 15:08:57.058655 11569 tablet_replica.cc:330] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f: stopping tablet replica
I20260811 15:08:57.058673 11570 raft_consensus.cc:2229] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:08:57.058735 11568 raft_consensus.cc:2229] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:57.058758 11569 raft_consensus.cc:2229] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:57.058827 11568 raft_consensus.cc:2258] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:57.058863 11569 raft_consensus.cc:2258] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:57.058857 11570 raft_consensus.cc:2258] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:57.059531 11568 ts_tablet_manager.cc:1905] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:57.059861 11570 ts_tablet_manager.cc:1905] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:57.061837 11568 ts_tablet_manager.cc:1918] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 15:08:57.061910 11568 log.cc:1196] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/18f81a0453ad40479afec6c162b4fefb
I20260811 15:08:57.062125 11568 ts_tablet_manager.cc:1939] T 18f81a0453ad40479afec6c162b4fefb P 982a70d4b9d44b51925104fde0947838: Deleting consensus metadata
I20260811 15:08:57.062674 11570 ts_tablet_manager.cc:1918] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 15:08:57.062747 11570 log.cc:1196] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-0/wal/wals/18f81a0453ad40479afec6c162b4fefb
I20260811 15:08:57.062834 10731 catalog_manager.cc:4841] TS 982a70d4b9d44b51925104fde0947838 (127.30.8.2:35591): tablet 18f81a0453ad40479afec6c162b4fefb (table test-table [id=65b5190a126e4831ac3c76baf6edcd6c]) successfully deleted
I20260811 15:08:57.063005 11570 ts_tablet_manager.cc:1939] T 18f81a0453ad40479afec6c162b4fefb P 88cf1c4770f64b97aef9328095fa61ce: Deleting consensus metadata
I20260811 15:08:57.063645 10736 catalog_manager.cc:4841] TS 88cf1c4770f64b97aef9328095fa61ce (127.30.8.1:44543): tablet 18f81a0453ad40479afec6c162b4fefb (table test-table [id=65b5190a126e4831ac3c76baf6edcd6c]) successfully deleted
I20260811 15:08:57.064391 11569 ts_tablet_manager.cc:1905] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:57.067116 11569 ts_tablet_manager.cc:1918] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 15:08:57.067190 11569 log.cc:1196] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/18f81a0453ad40479afec6c162b4fefb
I20260811 15:08:57.067430 11569 ts_tablet_manager.cc:1939] T 18f81a0453ad40479afec6c162b4fefb P 79314a19dd6643c286c38475262d781f: Deleting consensus metadata
I20260811 15:08:57.068192 10734 catalog_manager.cc:4841] TS 79314a19dd6643c286c38475262d781f (127.30.8.3:44281): tablet 18f81a0453ad40479afec6c162b4fefb (table test-table [id=65b5190a126e4831ac3c76baf6edcd6c]) successfully deleted
Aug 11 15:08:57 dist-test-slave-1vmg krb5kdc[10682](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 15:08:57 dist-test-slave-1vmg krb5kdc[10682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:08:57 dist-test-slave-1vmg krb5kdc[10682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 15:08:57.093492 10830 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:08:57.086448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54314 (local address 127.30.8.62:44939)
0811 15:08:57.086605 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:57.086610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:57.086786 (+ 176us) server_negotiation.cc:407] Connection header received
0811 15:08:57.087036 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:57.087039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:57.087086 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:57.087157 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:57.088099 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.088620 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.089774 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.090009 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.090083 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:57.093349 (+ 3266us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54314: BlockingRecv error: recv got EOF from 127.0.0.1:54314 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 15:08:57 dist-test-slave-1vmg krb5kdc[10682](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM
I20260811 15:08:57.103509 10830 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:57.095091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54322 (local address 127.30.8.62:44939)
0811 15:08:57.095198 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:57.095201 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:57.095422 (+ 221us) server_negotiation.cc:407] Connection header received
0811 15:08:57.095497 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:57.095508 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:57.095559 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:57.095650 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:08:57.096167 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.096947 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.097680 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.098012 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.098119 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:57.100489 (+ 2370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:08:57.100492 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:08:57.100494 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:08:57.100518 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:08:57.101901 (+ 1383us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:57.102389 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:57.102391 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:57.102393 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:57.102464 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:08:57.102738 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:08:57.102740 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:08:57.102741 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:08:57.102909 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:08:57.103049 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:57.103287 (+ 238us) server_negotiation.cc:299] Negotiation successful
0811 15:08:57.103382 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 15:08:57.106272 10754 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54322:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:08:57.106791 10754 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:08:57.111510 10968 tablet_service.cc:1467] Processing CreateTablet for tablet 937e4305a9f147e78350729decaa0603 (DEFAULT_TABLE table=test-table [id=2d57d0d0614e4af0bb394f6e629c60d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:57.111585 11178 tablet_service.cc:1467] Processing CreateTablet for tablet 937e4305a9f147e78350729decaa0603 (DEFAULT_TABLE table=test-table [id=2d57d0d0614e4af0bb394f6e629c60d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:57.111747 11394 tablet_service.cc:1467] Processing CreateTablet for tablet 937e4305a9f147e78350729decaa0603 (DEFAULT_TABLE table=test-table [id=2d57d0d0614e4af0bb394f6e629c60d2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:57.111824 10968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 937e4305a9f147e78350729decaa0603. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:57.111882 11178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 937e4305a9f147e78350729decaa0603. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:57.111987 11394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 937e4305a9f147e78350729decaa0603. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:57.114264 11535 tablet_bootstrap.cc:492] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838: Bootstrap starting.
I20260811 15:08:57.114261 11534 tablet_bootstrap.cc:492] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce: Bootstrap starting.
I20260811 15:08:57.114504 11532 tablet_bootstrap.cc:492] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f: Bootstrap starting.
I20260811 15:08:57.115167 11534 tablet_bootstrap.cc:654] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:57.115535 11535 tablet_bootstrap.cc:654] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:57.115871 11532 tablet_bootstrap.cc:654] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:57.117072 11535 tablet_bootstrap.cc:492] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838: No bootstrap required, opened a new log
I20260811 15:08:57.117179 11535 ts_tablet_manager.cc:1397] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260811 15:08:57.117599 11535 raft_consensus.cc:348] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:57.117658 11534 tablet_bootstrap.cc:492] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce: No bootstrap required, opened a new log
I20260811 15:08:57.117698 11535 raft_consensus.cc:374] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:57.117728 11535 raft_consensus.cc:729] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982a70d4b9d44b51925104fde0947838, State: Initialized, Role: FOLLOWER
I20260811 15:08:57.117733 11534 ts_tablet_manager.cc:1397] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260811 15:08:57.117848 11535 consensus_queue.cc:260] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:57.118113 11535 ts_tablet_manager.cc:1428] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 15:08:57.118090 11534 raft_consensus.cc:348] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:57.118188 11534 raft_consensus.cc:374] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:57.118216 11534 raft_consensus.cc:729] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88cf1c4770f64b97aef9328095fa61ce, State: Initialized, Role: FOLLOWER
I20260811 15:08:57.118322 11534 consensus_queue.cc:260] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:57.118587 11534 ts_tablet_manager.cc:1428] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:57.118953 11532 tablet_bootstrap.cc:492] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f: No bootstrap required, opened a new log
I20260811 15:08:57.119038 11532 ts_tablet_manager.cc:1397] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 15:08:57.119436 11532 raft_consensus.cc:348] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:57.119537 11532 raft_consensus.cc:374] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:57.119570 11532 raft_consensus.cc:729] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79314a19dd6643c286c38475262d781f, State: Initialized, Role: FOLLOWER
I20260811 15:08:57.119701 11532 consensus_queue.cc:260] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:57.119987 11532 ts_tablet_manager.cc:1428] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:57.297313 11541 raft_consensus.cc:482] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:08:57.297478 11541 raft_consensus.cc:504] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:57.298650 11541 leader_election.cc:288] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88cf1c4770f64b97aef9328095fa61ce (127.30.8.1:44543), 79314a19dd6643c286c38475262d781f (127.30.8.3:44281)
I20260811 15:08:57.301367 11561 raft_consensus.cc:482] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:08:57.301486 11561 raft_consensus.cc:504] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:57.301898 11561 leader_election.cc:288] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 982a70d4b9d44b51925104fde0947838 (127.30.8.2:35591), 79314a19dd6643c286c38475262d781f (127.30.8.3:44281)
I20260811 15:08:57.302273 11207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "937e4305a9f147e78350729decaa0603" candidate_uuid: "88cf1c4770f64b97aef9328095fa61ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982a70d4b9d44b51925104fde0947838" is_pre_election: true
I20260811 15:08:57.302407 11207 raft_consensus.cc:2454] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88cf1c4770f64b97aef9328095fa61ce in term 0.
I20260811 15:08:57.302415 11426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "937e4305a9f147e78350729decaa0603" candidate_uuid: "88cf1c4770f64b97aef9328095fa61ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79314a19dd6643c286c38475262d781f" is_pre_election: true
I20260811 15:08:57.302546 11426 raft_consensus.cc:2454] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 88cf1c4770f64b97aef9328095fa61ce in term 0.
I20260811 15:08:57.302700 10888 leader_election.cc:302] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 88cf1c4770f64b97aef9328095fa61ce, 982a70d4b9d44b51925104fde0947838; no voters:
I20260811 15:08:57.302868 11561 raft_consensus.cc:2781] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:08:57.302956 11561 raft_consensus.cc:482] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:08:57.303007 11561 raft_consensus.cc:3037] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:57.303649 11639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:57.298953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:44543 (local address 127.30.8.2:55609)
0811 15:08:57.299368 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:57.299381 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:57.299495 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:57.299721 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:57.299724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:57.299753 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:57.299843 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:57.299850 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.300793 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.300796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:57.302291 (+ 1495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:57.302299 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.303231 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.303234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:57.303319 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:57.303343 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 15:08:57.303417 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 15:08:57.303489 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"mutex_wait_us":259,"thread_start_us":141,"threads_started":1}
I20260811 15:08:57.303658 11530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:57.299026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:55609 (local address 127.30.8.1:44543)
0811 15:08:57.299372 (+ 346us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:57.299376 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:57.299458 (+ 82us) server_negotiation.cc:407] Connection header received
0811 15:08:57.299591 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:57.299595 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:57.299639 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:57.299700 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:57.299953 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.300688 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.302404 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.302976 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.303057 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:57.303134 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:57.303455 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 15:08:57.303542 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269}
I20260811 15:08:57.304008 10993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "937e4305a9f147e78350729decaa0603" candidate_uuid: "982a70d4b9d44b51925104fde0947838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88cf1c4770f64b97aef9328095fa61ce" is_pre_election: true
I20260811 15:08:57.304116 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:57.299154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:49573 (local address 127.30.8.3:44281)
0811 15:08:57.299298 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:57.299303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:57.299320 (+ 17us) server_negotiation.cc:407] Connection header received
0811 15:08:57.299465 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:57.299469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:57.299510 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:57.299573 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:08:57.299821 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.300634 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.302287 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.303355 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.303451 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:57.303512 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:57.303923 (+ 411us) server_negotiation.cc:299] Negotiation successful
0811 15:08:57.303998 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 15:08:57.304142 11638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:57.298950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:44281 (local address 127.30.8.2:49573)
0811 15:08:57.299228 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:57.299245 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:57.299362 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:57.299593 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:57.299596 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:57.299612 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:08:57.299712 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:57.299717 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.300755 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.300759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:57.302152 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:57.302163 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.303599 (+ 1436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.303603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:57.303688 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:57.303823 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 15:08:57.303894 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 15:08:57.303968 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":3,"thread_start_us":86,"threads_started":1}
I20260811 15:08:57.304328 11561 raft_consensus.cc:504] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:57.304593 10993 raft_consensus.cc:2379] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 982a70d4b9d44b51925104fde0947838 in current term 1: Already voted for candidate 88cf1c4770f64b97aef9328095fa61ce in this term.
I20260811 15:08:57.304623 11426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "937e4305a9f147e78350729decaa0603" candidate_uuid: "982a70d4b9d44b51925104fde0947838" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79314a19dd6643c286c38475262d781f" is_pre_election: true
I20260811 15:08:57.304705 11561 leader_election.cc:288] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [CANDIDATE]: Term 1 election: Requested vote from peers 982a70d4b9d44b51925104fde0947838 (127.30.8.2:35591), 79314a19dd6643c286c38475262d781f (127.30.8.3:44281)
I20260811 15:08:57.304749 11426 raft_consensus.cc:2454] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 982a70d4b9d44b51925104fde0947838 in term 0.
I20260811 15:08:57.305138 11106 leader_election.cc:302] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 79314a19dd6643c286c38475262d781f, 982a70d4b9d44b51925104fde0947838; no voters: 88cf1c4770f64b97aef9328095fa61ce
I20260811 15:08:57.305152 11207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "937e4305a9f147e78350729decaa0603" candidate_uuid: "88cf1c4770f64b97aef9328095fa61ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982a70d4b9d44b51925104fde0947838"
I20260811 15:08:57.305260 11426 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "937e4305a9f147e78350729decaa0603" candidate_uuid: "88cf1c4770f64b97aef9328095fa61ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "79314a19dd6643c286c38475262d781f"
I20260811 15:08:57.305302 11207 raft_consensus.cc:3037] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:57.305353 11426 raft_consensus.cc:3037] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:57.306389 11426 raft_consensus.cc:2454] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88cf1c4770f64b97aef9328095fa61ce in term 1.
I20260811 15:08:57.306411 11207 raft_consensus.cc:2454] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 88cf1c4770f64b97aef9328095fa61ce in term 1.
I20260811 15:08:57.306524 11541 raft_consensus.cc:2741] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260811 15:08:57.306715 10890 leader_election.cc:302] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79314a19dd6643c286c38475262d781f, 88cf1c4770f64b97aef9328095fa61ce; no voters:
I20260811 15:08:57.307358 11561 raft_consensus.cc:2781] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:57.307441 11561 raft_consensus.cc:686] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [term 1 LEADER]: Becoming Leader. State: Replica: 88cf1c4770f64b97aef9328095fa61ce, State: Running, Role: LEADER
I20260811 15:08:57.307590 11561 consensus_queue.cc:237] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:57.309139 10754 catalog_manager.cc:5495] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce reported cstate change: term changed from 0 to 1, leader changed from <none> to 88cf1c4770f64b97aef9328095fa61ce (127.30.8.1). New cstate: current_term: 1 leader_uuid: "88cf1c4770f64b97aef9328095fa61ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } health_report { overall_health: UNKNOWN } } }
I20260811 15:08:57.357774 11530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:57.355217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42336 (local address 127.30.8.1:44543)
0811 15:08:57.355341 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:57.355345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:57.355359 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:08:57.355445 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:57.355447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:57.355493 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:57.355552 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:57.355726 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.356233 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.356981 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.357188 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.357252 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:57.357423 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:57.357488 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:57.357534 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:57.357595 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 15:08:57.357635 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 15:08:57.359747 11207 raft_consensus.cc:1261] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838 [term 1 FOLLOWER]: Refusing update from remote peer 88cf1c4770f64b97aef9328095fa61ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:57.359764 11426 raft_consensus.cc:1261] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f [term 1 FOLLOWER]: Refusing update from remote peer 88cf1c4770f64b97aef9328095fa61ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:57.360249 11542 consensus_queue.cc:1035] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:57.360401 11561 consensus_queue.cc:1035] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:57.366851 10755 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:57.367027 10755 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:57.369392 10755 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49: Sending DeleteTablet for 3 replicas of tablet 937e4305a9f147e78350729decaa0603
I20260811 15:08:57.370029 11178 tablet_service.cc:1514] Processing DeleteTablet for tablet 937e4305a9f147e78350729decaa0603 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:57 UTC) from {username='oryx', principal='oryx/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:34838
I20260811 15:08:57.370114 11394 tablet_service.cc:1514] Processing DeleteTablet for tablet 937e4305a9f147e78350729decaa0603 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:57 UTC) from {username='oryx', principal='oryx/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:53276
I20260811 15:08:57.370154 10968 tablet_service.cc:1514] Processing DeleteTablet for tablet 937e4305a9f147e78350729decaa0603 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:57 UTC) from {username='oryx', principal='oryx/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:42314
I20260811 15:08:57.370265 11569 tablet_replica.cc:330] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f: stopping tablet replica
I20260811 15:08:57.370321 11570 tablet_replica.cc:330] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce: stopping tablet replica
I20260811 15:08:57.370334 11568 tablet_replica.cc:330] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838: stopping tablet replica
I20260811 15:08:57.370381 11569 raft_consensus.cc:2229] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:57.370432 11568 raft_consensus.cc:2229] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:57.370435 11570 raft_consensus.cc:2229] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:08:57.370517 11569 raft_consensus.cc:2258] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:57.370556 11568 raft_consensus.cc:2258] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:57.370610 11570 raft_consensus.cc:2258] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:57.370271 11393 tablet_service.cc:1514] Processing DeleteTablet for tablet 937e4305a9f147e78350729decaa0603 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:57 UTC) from {username='oryx', principal='oryx/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:53276
I20260811 15:08:57.371214 11569 ts_tablet_manager.cc:1905] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:57.371268 11570 ts_tablet_manager.cc:1905] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:57.371426 11568 ts_tablet_manager.cc:1905] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:57.372066 10754 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54322:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:08:57.372464 10754 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:08:57.373600 11570 ts_tablet_manager.cc:1918] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:08:57.373603 11569 ts_tablet_manager.cc:1918] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:08:57.373678 11570 log.cc:1196] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-0/wal/wals/937e4305a9f147e78350729decaa0603
I20260811 15:08:57.373678 11569 log.cc:1196] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/937e4305a9f147e78350729decaa0603
I20260811 15:08:57.373806 11568 ts_tablet_manager.cc:1918] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:08:57.373884 11568 log.cc:1196] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/937e4305a9f147e78350729decaa0603
I20260811 15:08:57.373926 11569 ts_tablet_manager.cc:1939] T 937e4305a9f147e78350729decaa0603 P 79314a19dd6643c286c38475262d781f: Deleting consensus metadata
I20260811 15:08:57.373927 11570 ts_tablet_manager.cc:1939] T 937e4305a9f147e78350729decaa0603 P 88cf1c4770f64b97aef9328095fa61ce: Deleting consensus metadata
I20260811 15:08:57.374130 11568 ts_tablet_manager.cc:1939] T 937e4305a9f147e78350729decaa0603 P 982a70d4b9d44b51925104fde0947838: Deleting consensus metadata
I20260811 15:08:57.374625 10734 catalog_manager.cc:4841] TS 79314a19dd6643c286c38475262d781f (127.30.8.3:44281): tablet 937e4305a9f147e78350729decaa0603 (table test-table [id=2d57d0d0614e4af0bb394f6e629c60d2]) successfully deleted
I20260811 15:08:57.374625 10736 catalog_manager.cc:4841] TS 88cf1c4770f64b97aef9328095fa61ce (127.30.8.1:44543): tablet 937e4305a9f147e78350729decaa0603 (table test-table [id=2d57d0d0614e4af0bb394f6e629c60d2]) successfully deleted
I20260811 15:08:57.374827 10731 catalog_manager.cc:4841] TS 982a70d4b9d44b51925104fde0947838 (127.30.8.2:35591): tablet 937e4305a9f147e78350729decaa0603 (table test-table [id=2d57d0d0614e4af0bb394f6e629c60d2]) successfully deleted
W20260811 15:08:57.375432 10734 catalog_manager.cc:4808] TS 79314a19dd6643c286c38475262d781f (127.30.8.3:44281): delete failed for tablet 937e4305a9f147e78350729decaa0603 because the tablet was not found. No further retry: Not found: Tablet not found: 937e4305a9f147e78350729decaa0603
I20260811 15:08:57.377605 10968 tablet_service.cc:1467] Processing CreateTablet for tablet 400c14a5854e40e2b6191901998abfbf (DEFAULT_TABLE table=test-table [id=c6a14237a1784c54ac68f1950fba36a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:57.377642 11178 tablet_service.cc:1467] Processing CreateTablet for tablet 400c14a5854e40e2b6191901998abfbf (DEFAULT_TABLE table=test-table [id=c6a14237a1784c54ac68f1950fba36a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:57.377696 11393 tablet_service.cc:1467] Processing CreateTablet for tablet 400c14a5854e40e2b6191901998abfbf (DEFAULT_TABLE table=test-table [id=c6a14237a1784c54ac68f1950fba36a7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:08:57.377897 11178 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 400c14a5854e40e2b6191901998abfbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:57.377897 10968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 400c14a5854e40e2b6191901998abfbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:57.377915 11393 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 400c14a5854e40e2b6191901998abfbf. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:08:57.379673 11535 tablet_bootstrap.cc:492] T 400c14a5854e40e2b6191901998abfbf P 982a70d4b9d44b51925104fde0947838: Bootstrap starting.
I20260811 15:08:57.380200 11532 tablet_bootstrap.cc:492] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f: Bootstrap starting.
I20260811 15:08:57.380200 11534 tablet_bootstrap.cc:492] T 400c14a5854e40e2b6191901998abfbf P 88cf1c4770f64b97aef9328095fa61ce: Bootstrap starting.
I20260811 15:08:57.380640 11535 tablet_bootstrap.cc:654] T 400c14a5854e40e2b6191901998abfbf P 982a70d4b9d44b51925104fde0947838: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:57.380947 11534 tablet_bootstrap.cc:654] T 400c14a5854e40e2b6191901998abfbf P 88cf1c4770f64b97aef9328095fa61ce: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:57.380947 11532 tablet_bootstrap.cc:654] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f: Neither blocks nor log segments found. Creating new log.
I20260811 15:08:57.382000 11534 tablet_bootstrap.cc:492] T 400c14a5854e40e2b6191901998abfbf P 88cf1c4770f64b97aef9328095fa61ce: No bootstrap required, opened a new log
I20260811 15:08:57.382099 11534 ts_tablet_manager.cc:1397] T 400c14a5854e40e2b6191901998abfbf P 88cf1c4770f64b97aef9328095fa61ce: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 15:08:57.382124 11535 tablet_bootstrap.cc:492] T 400c14a5854e40e2b6191901998abfbf P 982a70d4b9d44b51925104fde0947838: No bootstrap required, opened a new log
I20260811 15:08:57.382215 11535 ts_tablet_manager.cc:1397] T 400c14a5854e40e2b6191901998abfbf P 982a70d4b9d44b51925104fde0947838: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260811 15:08:57.382355 11532 tablet_bootstrap.cc:492] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f: No bootstrap required, opened a new log
I20260811 15:08:57.382431 11532 ts_tablet_manager.cc:1397] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 15:08:57.382496 11534 raft_consensus.cc:348] T 400c14a5854e40e2b6191901998abfbf P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:57.382601 11534 raft_consensus.cc:374] T 400c14a5854e40e2b6191901998abfbf P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:57.382630 11534 raft_consensus.cc:729] T 400c14a5854e40e2b6191901998abfbf P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88cf1c4770f64b97aef9328095fa61ce, State: Initialized, Role: FOLLOWER
I20260811 15:08:57.382620 11535 raft_consensus.cc:348] T 400c14a5854e40e2b6191901998abfbf P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:57.382738 11535 raft_consensus.cc:374] T 400c14a5854e40e2b6191901998abfbf P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:57.382768 11535 raft_consensus.cc:729] T 400c14a5854e40e2b6191901998abfbf P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 982a70d4b9d44b51925104fde0947838, State: Initialized, Role: FOLLOWER
I20260811 15:08:57.382754 11534 consensus_queue.cc:260] T 400c14a5854e40e2b6191901998abfbf P 88cf1c4770f64b97aef9328095fa61ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:57.382829 11532 raft_consensus.cc:348] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:57.382874 11535 consensus_queue.cc:260] T 400c14a5854e40e2b6191901998abfbf P 982a70d4b9d44b51925104fde0947838 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:57.382946 11532 raft_consensus.cc:374] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:08:57.382980 11532 raft_consensus.cc:729] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 79314a19dd6643c286c38475262d781f, State: Initialized, Role: FOLLOWER
I20260811 15:08:57.383049 11534 ts_tablet_manager.cc:1428] T 400c14a5854e40e2b6191901998abfbf P 88cf1c4770f64b97aef9328095fa61ce: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:57.383095 11532 consensus_queue.cc:260] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:57.383131 11535 ts_tablet_manager.cc:1428] T 400c14a5854e40e2b6191901998abfbf P 982a70d4b9d44b51925104fde0947838: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 15:08:57.383338 11532 ts_tablet_manager.cc:1428] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 15:08:57.423785 11601 raft_consensus.cc:482] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:08:57.423931 11601 raft_consensus.cc:504] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:57.424352 11601 leader_election.cc:288] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 88cf1c4770f64b97aef9328095fa61ce (127.30.8.1:44543), 982a70d4b9d44b51925104fde0947838 (127.30.8.2:35591)
I20260811 15:08:57.424806 11207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "400c14a5854e40e2b6191901998abfbf" candidate_uuid: "79314a19dd6643c286c38475262d781f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982a70d4b9d44b51925104fde0947838" is_pre_election: true
I20260811 15:08:57.424806 10993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "400c14a5854e40e2b6191901998abfbf" candidate_uuid: "79314a19dd6643c286c38475262d781f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88cf1c4770f64b97aef9328095fa61ce" is_pre_election: true
I20260811 15:08:57.424959 11207 raft_consensus.cc:2454] T 400c14a5854e40e2b6191901998abfbf P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79314a19dd6643c286c38475262d781f in term 0.
I20260811 15:08:57.424959 10993 raft_consensus.cc:2454] T 400c14a5854e40e2b6191901998abfbf P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 79314a19dd6643c286c38475262d781f in term 0.
I20260811 15:08:57.425261 11321 leader_election.cc:302] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 79314a19dd6643c286c38475262d781f, 88cf1c4770f64b97aef9328095fa61ce, 982a70d4b9d44b51925104fde0947838; no voters:
I20260811 15:08:57.425441 11601 raft_consensus.cc:2781] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:08:57.425501 11601 raft_consensus.cc:482] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:08:57.425534 11601 raft_consensus.cc:3037] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:57.426535 11601 raft_consensus.cc:504] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:57.426859 11601 leader_election.cc:288] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [CANDIDATE]: Term 1 election: Requested vote from peers 88cf1c4770f64b97aef9328095fa61ce (127.30.8.1:44543), 982a70d4b9d44b51925104fde0947838 (127.30.8.2:35591)
I20260811 15:08:57.427215 10993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "400c14a5854e40e2b6191901998abfbf" candidate_uuid: "79314a19dd6643c286c38475262d781f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88cf1c4770f64b97aef9328095fa61ce"
I20260811 15:08:57.427237 11207 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "400c14a5854e40e2b6191901998abfbf" candidate_uuid: "79314a19dd6643c286c38475262d781f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "982a70d4b9d44b51925104fde0947838"
I20260811 15:08:57.427331 10993 raft_consensus.cc:3037] T 400c14a5854e40e2b6191901998abfbf P 88cf1c4770f64b97aef9328095fa61ce [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:57.427343 11207 raft_consensus.cc:3037] T 400c14a5854e40e2b6191901998abfbf P 982a70d4b9d44b51925104fde0947838 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:08:57.428196 11207 raft_consensus.cc:2454] T 400c14a5854e40e2b6191901998abfbf P 982a70d4b9d44b51925104fde0947838 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79314a19dd6643c286c38475262d781f in term 1.
I20260811 15:08:57.428265 10993 raft_consensus.cc:2454] T 400c14a5854e40e2b6191901998abfbf P 88cf1c4770f64b97aef9328095fa61ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 79314a19dd6643c286c38475262d781f in term 1.
I20260811 15:08:57.428486 11321 leader_election.cc:302] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 79314a19dd6643c286c38475262d781f, 982a70d4b9d44b51925104fde0947838; no voters:
I20260811 15:08:57.428685 11601 raft_consensus.cc:2781] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:08:57.428773 11601 raft_consensus.cc:686] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [term 1 LEADER]: Becoming Leader. State: Replica: 79314a19dd6643c286c38475262d781f, State: Running, Role: LEADER
I20260811 15:08:57.428913 11601 consensus_queue.cc:237] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } }
I20260811 15:08:57.430560 10754 catalog_manager.cc:5495] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f reported cstate change: term changed from 0 to 1, leader changed from <none> to 79314a19dd6643c286c38475262d781f (127.30.8.3). New cstate: current_term: 1 leader_uuid: "79314a19dd6643c286c38475262d781f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "79314a19dd6643c286c38475262d781f" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 44281 } health_report { overall_health: HEALTHY } } }
I20260811 15:08:57.456890 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:57.453962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53294 (local address 127.30.8.3:44281)
0811 15:08:57.454068 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:57.454074 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:57.454117 (+ 43us) server_negotiation.cc:407] Connection header received
0811 15:08:57.454257 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:57.454260 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:57.454304 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:57.454363 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:57.454589 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.455144 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.455934 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.456157 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.456243 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:57.456472 (+ 229us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:57.456551 (+ 79us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:57.456607 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:57.456702 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 15:08:57.456754 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 15:08:57.465080 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:57.462391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:44823 (local address 127.30.8.3:44281)
0811 15:08:57.462476 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0811 15:08:57.462480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:08:57.462514 (+ 34us) server_negotiation.cc:407] Connection header received
0811 15:08:57.462640 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:08:57.462643 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:08:57.462677 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:08:57.462733 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:08:57.462967 (+ 234us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.463440 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.464202 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.464396 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.464476 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:57.464684 (+ 208us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:08:57.464748 (+ 64us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:08:57.464803 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 15:08:57.464885 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 15:08:57.464927 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":33}
I20260811 15:08:57.465080 11611 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:57.462310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:44281 (local address 127.30.8.3:44823)
0811 15:08:57.462448 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:57.462462 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:08:57.462557 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:08:57.462746 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:08:57.462748 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:08:57.462755 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 15:08:57.462862 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:57.462869 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.463533 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.463536 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:57.464100 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:08:57.464108 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:08:57.464483 (+ 375us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:08:57.464487 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:08:57.464562 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:08:57.464588 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 15:08:57.464815 (+ 227us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 15:08:57.464821 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 15:08:57.464877 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 15:08:57.464957 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":3}
I20260811 15:08:57.470216 10993 raft_consensus.cc:1261] T 400c14a5854e40e2b6191901998abfbf P 88cf1c4770f64b97aef9328095fa61ce [term 1 FOLLOWER]: Refusing update from remote peer 79314a19dd6643c286c38475262d781f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:57.470218 11207 raft_consensus.cc:1261] T 400c14a5854e40e2b6191901998abfbf P 982a70d4b9d44b51925104fde0947838 [term 1 FOLLOWER]: Refusing update from remote peer 79314a19dd6643c286c38475262d781f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:08:57.470690 11593 consensus_queue.cc:1035] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [LEADER]: Connected to new peer: Peer: permanent_uuid: "982a70d4b9d44b51925104fde0947838" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 35591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:57.470839 11601 consensus_queue.cc:1035] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [LEADER]: Connected to new peer: Peer: permanent_uuid: "88cf1c4770f64b97aef9328095fa61ce" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44543 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:08:57.504169 10748 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:57.504334 10748 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54322:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:08:57.506342 10748 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bd7b8f4a58ca4ed8bea296e790287e49: Sending DeleteTablet for 3 replicas of tablet 400c14a5854e40e2b6191901998abfbf
I20260811 15:08:57.506906 10968 tablet_service.cc:1514] Processing DeleteTablet for tablet 400c14a5854e40e2b6191901998abfbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:57 UTC) from {username='oryx', principal='oryx/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:42314
I20260811 15:08:57.507182 11570 tablet_replica.cc:330] T 400c14a5854e40e2b6191901998abfbf P 88cf1c4770f64b97aef9328095fa61ce: stopping tablet replica
I20260811 15:08:57.507308 11570 raft_consensus.cc:2229] T 400c14a5854e40e2b6191901998abfbf P 88cf1c4770f64b97aef9328095fa61ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:57.507431 11570 raft_consensus.cc:2258] T 400c14a5854e40e2b6191901998abfbf P 88cf1c4770f64b97aef9328095fa61ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:57.507611 11178 tablet_service.cc:1514] Processing DeleteTablet for tablet 400c14a5854e40e2b6191901998abfbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:57 UTC) from {username='oryx', principal='oryx/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:34838
I20260811 15:08:57.507794 11568 tablet_replica.cc:330] T 400c14a5854e40e2b6191901998abfbf P 982a70d4b9d44b51925104fde0947838: stopping tablet replica
I20260811 15:08:57.507774 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 10862
I20260811 15:08:57.507901 11568 raft_consensus.cc:2229] T 400c14a5854e40e2b6191901998abfbf P 982a70d4b9d44b51925104fde0947838 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:08:57.508014 11568 raft_consensus.cc:2258] T 400c14a5854e40e2b6191901998abfbf P 982a70d4b9d44b51925104fde0947838 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:57.517657 11394 tablet_service.cc:1514] Processing DeleteTablet for tablet 400c14a5854e40e2b6191901998abfbf with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:08:57 UTC) from {username='oryx', principal='oryx/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:53276
I20260811 15:08:57.517892 11569 tablet_replica.cc:330] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f: stopping tablet replica
I20260811 15:08:57.518033 11569 raft_consensus.cc:2229] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:08:57.518213 11568 ts_tablet_manager.cc:1905] T 400c14a5854e40e2b6191901998abfbf P 982a70d4b9d44b51925104fde0947838: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:57.518432 11569 raft_consensus.cc:2258] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:08:57.519959 11569 ts_tablet_manager.cc:1905] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:08:57.520887 11568 ts_tablet_manager.cc:1918] T 400c14a5854e40e2b6191901998abfbf P 982a70d4b9d44b51925104fde0947838: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 15:08:57.520961 11568 log.cc:1196] T 400c14a5854e40e2b6191901998abfbf P 982a70d4b9d44b51925104fde0947838: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/400c14a5854e40e2b6191901998abfbf
I20260811 15:08:57.521229 11568 ts_tablet_manager.cc:1939] T 400c14a5854e40e2b6191901998abfbf P 982a70d4b9d44b51925104fde0947838: Deleting consensus metadata
I20260811 15:08:57.521996 10731 catalog_manager.cc:4841] TS 982a70d4b9d44b51925104fde0947838 (127.30.8.2:35591): tablet 400c14a5854e40e2b6191901998abfbf (table test-table [id=c6a14237a1784c54ac68f1950fba36a7]) successfully deleted
I20260811 15:08:57.522363 11569 ts_tablet_manager.cc:1918] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 15:08:57.522424 11569 log.cc:1196] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/400c14a5854e40e2b6191901998abfbf
I20260811 15:08:57.522598 11569 ts_tablet_manager.cc:1939] T 400c14a5854e40e2b6191901998abfbf P 79314a19dd6643c286c38475262d781f: Deleting consensus metadata
I20260811 15:08:57.523208 10734 catalog_manager.cc:4841] TS 79314a19dd6643c286c38475262d781f (127.30.8.3:44281): tablet 400c14a5854e40e2b6191901998abfbf (table test-table [id=c6a14237a1784c54ac68f1950fba36a7]) successfully deleted
W20260811 15:08:57.537078 10736 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.8.1:44543 (error 108)
I20260811 15:08:57.537256 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 11077
I20260811 15:08:57.543197 11662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:08:57.540661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:44543 (local address 127.0.0.1:42338)
0811 15:08:57.542953 (+ 2292us) negotiation.cc:107] Waiting for socket to connect
0811 15:08:57.543041 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.1:44543: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2175,"thread_start_us":61,"threads_started":1}
W20260811 15:08:57.543388 10736 catalog_manager.cc:4584] TS 88cf1c4770f64b97aef9328095fa61ce (127.30.8.1:44543): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 400c14a5854e40e2b6191901998abfbf: Network error: Client connection negotiation failed: client connection to 127.30.8.1:44543: connect: Connection refused (error 111)
I20260811 15:08:57.549584 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 11299
I20260811 15:08:57.568828 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 10707
2026-08-11T15:08:57Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3602 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:08:57 dist-test-slave-1vmg krb5kdc[11666](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:08:57 dist-test-slave-1vmg krb5kdc[11666](info): set up 2 sockets
Aug 11 15:08:57 dist-test-slave-1vmg krb5kdc[11666](info): commencing operation
krb5kdc: starting...
W20260811 15:08:58.745695 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:08:58 dist-test-slave-1vmg krb5kdc[11666](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:08:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:08:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:08:58.882583 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:36899
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:40403
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:36899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:08:59.011976 11681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:08:59.012305 11681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:08:59.012368 11681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:08:59.016562 11681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:08:59.016649 11681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:08:59.016678 11681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:08:59.016700 11681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:08:59.016728 11681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:08:59.022547 11681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:40403
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:36899
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:36899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:08:59.023833 11681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:08:59.024933 11681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:08:59.034063 11692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:59.034060 11690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:08:59.034107 11689 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:08:59.034619 11681 server_base.cc:1034] running on GCE node
I20260811 15:08:59.035605 11681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:08:59.037387 11681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:08:59.038590 11681 hybrid_clock.cc:648] HybridClock initialized: now 1786460939038559 us; error 54 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T15:08:59Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1478 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:08:59 dist-test-slave-1vmg krb5kdc[11702](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:08:59 dist-test-slave-1vmg krb5kdc[11702](info): set up 2 sockets
Aug 11 15:08:59 dist-test-slave-1vmg krb5kdc[11702](info): commencing operation
krb5kdc: starting...
W20260811 15:09:00.206410 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:09:00 dist-test-slave-1vmg krb5kdc[11702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:09:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:09:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:09:00.323529 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:34091
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:40395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:34091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:09:00.433712 11718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:09:00.433969 11718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:09:00.434026 11718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:09:00.437997 11718 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:09:00.438054 11718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:09:00.438081 11718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:09:00.438102 11718 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:09:00.438127 11718 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:09:00.443310 11718 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:40395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:34091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:34091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:09:00.444308 11718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:09:00.445292 11718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 15:09:00.453327 11718 server_base.cc:1034] running on GCE node
W20260811 15:09:00.453289 11725 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:00.453312 11723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:00.453287 11727 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:00.454128 11718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:09:00.455227 11718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:09:00.456380 11718 hybrid_clock.cc:648] HybridClock initialized: now 1786460940456369 us; error 26 us; skew 500 ppm
Aug 11 15:09:00 dist-test-slave-1vmg krb5kdc[11702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460940, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:09:00.458958 11718 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:09:00.459815 11718 webserver.cc:460] Webserver started at http://127.30.8.62:44009/ using document root <none> and password file <none>
I20260811 15:09:00.460325 11718 fs_manager.cc:359] Metadata directory not provided
I20260811 15:09:00.460382 11718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:09:00.460552 11718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:09:00.462481 11718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "bfc279b5adcf4ad7a07020f1efb590b5"
format_stamp: "Formatted at 2026-08-11 15:09:00 on dist-test-slave-1vmg"
server_key: "df9937669317a92b1c19b57922a7cfe6"
server_key_iv: "53b9763cb5c0523d6f1def0cef3920fb"
server_key_version: "encryptionkey@0"
I20260811 15:09:00.463025 11718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "bfc279b5adcf4ad7a07020f1efb590b5"
format_stamp: "Formatted at 2026-08-11 15:09:00 on dist-test-slave-1vmg"
server_key: "df9937669317a92b1c19b57922a7cfe6"
server_key_iv: "53b9763cb5c0523d6f1def0cef3920fb"
server_key_version: "encryptionkey@0"
I20260811 15:09:00.466495 11718 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 15:09:00.468765 11740 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:00.469885 11718 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 15:09:00.470001 11718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "bfc279b5adcf4ad7a07020f1efb590b5"
format_stamp: "Formatted at 2026-08-11 15:09:00 on dist-test-slave-1vmg"
server_key: "df9937669317a92b1c19b57922a7cfe6"
server_key_iv: "53b9763cb5c0523d6f1def0cef3920fb"
server_key_version: "encryptionkey@0"
I20260811 15:09:00.470093 11718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:09:00.495325 11718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:09:00.498315 11718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:09:00.498510 11718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:09:00.505506 11718 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:34091
I20260811 15:09:00.505527 11826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:34091 every 8 connection(s)
I20260811 15:09:00.506523 11718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:09:00.509166 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 11718
I20260811 15:09:00.509306 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:09:00.509632 30752 external_mini_cluster.cc:1428] Setting key f5b31d4cb93d830136339f53088de5cc
I20260811 15:09:00.509708 11827 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:09:00.516920 11827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5: Bootstrap starting.
Aug 11 15:09:00 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:09:00.519758 11827 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5: Neither blocks nor log segments found. Creating new log.
I20260811 15:09:00.520503 11827 log.cc:824] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5: Log is configured to *not* fsync() on all Append() calls
I20260811 15:09:00.522780 11827 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5: No bootstrap required, opened a new log
I20260811 15:09:00.524195 11833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:00.511076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43224 (local address 127.30.8.62:34091)
0811 15:09:00.511489 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:00.511511 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:00.511548 (+ 37us) server_negotiation.cc:407] Connection header received
0811 15:09:00.512442 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:00.512459 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:00.512789 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:00.513068 (+ 279us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:00.513957 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:00.514988 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:00.516087 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:00.516562 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:00.516675 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:00.519498 (+ 2823us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:09:00.519504 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:09:00.519518 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:09:00.519547 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:09:00.521345 (+ 1798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:09:00.521795 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:09:00.521799 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:09:00.521804 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:09:00.521950 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:09:00.522250 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:09:00.522252 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:09:00.522253 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:09:00.522662 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:09:00.522809 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:00.523146 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 15:09:00.523453 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":273,"thread_start_us":122,"threads_started":1}
I20260811 15:09:00.526885 11827 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc279b5adcf4ad7a07020f1efb590b5" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34091 } }
I20260811 15:09:00.527149 11827 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:09:00.527208 11827 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfc279b5adcf4ad7a07020f1efb590b5, State: Initialized, Role: FOLLOWER
I20260811 15:09:00.527628 11827 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc279b5adcf4ad7a07020f1efb590b5" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34091 } }
I20260811 15:09:00.527755 11827 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:09:00.527817 11827 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:09:00.527909 11827 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:09:00.528847 11827 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc279b5adcf4ad7a07020f1efb590b5" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34091 } }
I20260811 15:09:00.529181 11827 leader_election.cc:302] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfc279b5adcf4ad7a07020f1efb590b5; no voters:
I20260811 15:09:00.529482 11827 leader_election.cc:288] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:09:00.529590 11835 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:09:00.529816 11835 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [term 1 LEADER]: Becoming Leader. State: Replica: bfc279b5adcf4ad7a07020f1efb590b5, State: Running, Role: LEADER
I20260811 15:09:00.530164 11835 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc279b5adcf4ad7a07020f1efb590b5" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34091 } }
I20260811 15:09:00.530702 11827 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:09:00.531867 11837 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfc279b5adcf4ad7a07020f1efb590b5. Latest consensus state: current_term: 1 leader_uuid: "bfc279b5adcf4ad7a07020f1efb590b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc279b5adcf4ad7a07020f1efb590b5" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34091 } } }
I20260811 15:09:00.532002 11837 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [sys.catalog]: This master's current role is: LEADER
I20260811 15:09:00.532279 11836 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfc279b5adcf4ad7a07020f1efb590b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc279b5adcf4ad7a07020f1efb590b5" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34091 } } }
I20260811 15:09:00.532363 11836 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [sys.catalog]: This master's current role is: LEADER
I20260811 15:09:00.533027 11848 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:09:00.536017 11848 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:09:00.541729 11848 catalog_manager.cc:1347] Generated new cluster ID: f9702c74f96a496484c9edaec2d94166
I20260811 15:09:00.541800 11848 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:09:00.555760 11848 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:09:00.556738 11848 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:09:00.562664 11848 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5: Generated new TSK 0
I20260811 15:09:00.563392 11848 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
I20260811 15:09:00.599998 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:34091
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:40395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:09:00.716816 11865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:09:00.717118 11865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:09:00.717162 11865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:09:00.721242 11865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:09:00.721326 11865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:09:00.721387 11865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:09:00.726837 11865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:40395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:34091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:09:00.728042 11865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:09:00.729049 11865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:09:00.738863 11875 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:00.738958 11874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:00.738863 11878 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:00.739238 11865 server_base.cc:1034] running on GCE node
I20260811 15:09:00.739800 11865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:09:00.740473 11865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:09:00.741647 11865 hybrid_clock.cc:648] HybridClock initialized: now 1786460940741627 us; error 40 us; skew 500 ppm
Aug 11 15:09:00 dist-test-slave-1vmg krb5kdc[11702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460940, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:09:00.744402 11865 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:09:00.745327 11865 webserver.cc:460] Webserver started at http://127.30.8.1:43677/ using document root <none> and password file <none>
I20260811 15:09:00.745908 11865 fs_manager.cc:359] Metadata directory not provided
I20260811 15:09:00.745977 11865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:09:00.746184 11865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:09:00.748332 11865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "ede0f9ef8a53435a9702e81726664d4e"
format_stamp: "Formatted at 2026-08-11 15:09:00 on dist-test-slave-1vmg"
server_key: "27aa328d957d9d00cf63f0a72a77c888"
server_key_iv: "739446d72d1ca91f421fd1c1c700ad7a"
server_key_version: "encryptionkey@0"
I20260811 15:09:00.748888 11865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "ede0f9ef8a53435a9702e81726664d4e"
format_stamp: "Formatted at 2026-08-11 15:09:00 on dist-test-slave-1vmg"
server_key: "27aa328d957d9d00cf63f0a72a77c888"
server_key_iv: "739446d72d1ca91f421fd1c1c700ad7a"
server_key_version: "encryptionkey@0"
I20260811 15:09:00.752959 11865 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:09:00.755455 11889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:00.756649 11865 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:09:00.756767 11865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "ede0f9ef8a53435a9702e81726664d4e"
format_stamp: "Formatted at 2026-08-11 15:09:00 on dist-test-slave-1vmg"
server_key: "27aa328d957d9d00cf63f0a72a77c888"
server_key_iv: "739446d72d1ca91f421fd1c1c700ad7a"
server_key_version: "encryptionkey@0"
I20260811 15:09:00.756870 11865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:09:00.776969 11865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:09:00.779965 11865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:09:00.780167 11865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:09:00.780793 11865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:09:00.781673 11865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:09:00.781724 11865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:00.781780 11865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:09:00.781808 11865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:00.790417 11865 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:38981
I20260811 15:09:00.790465 12066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:38981 every 8 connection(s)
I20260811 15:09:00.791388 11865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:09:00.797309 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 11865
I20260811 15:09:00.797446 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:09:00.797770 30752 external_mini_cluster.cc:1428] Setting key 0d8018a7bf57b72ae549da8d005de2a2
Aug 11 15:09:00 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460940, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:09:00.805303 11833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:00.793258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:50475 (local address 127.30.8.62:34091)
0811 15:09:00.793396 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:00.793399 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:00.794231 (+ 832us) server_negotiation.cc:407] Connection header received
0811 15:09:00.795488 (+ 1257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:00.795491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:00.795542 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:00.795613 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:00.796760 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:00.797293 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:00.798756 (+ 1463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:00.799016 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:00.799122 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:00.801876 (+ 2754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:09:00.801879 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:09:00.801881 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:09:00.801907 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:09:00.803249 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:09:00.803748 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:09:00.803752 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:09:00.803754 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:09:00.803831 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:09:00.804158 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:09:00.804162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:09:00.804164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:09:00.804360 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:09:00.804447 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:00.804984 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 15:09:00.805118 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 15:09:00.806316 12070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:00.793627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.1:50475)
0811 15:09:00.794074 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:00.794117 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:00.795296 (+ 1179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:00.795756 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:00.795763 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:00.796123 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:00.796635 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:00.796649 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:00.797448 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:00.797452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:00.798642 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:00.798656 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:00.799200 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:00.799205 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:00.799402 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:00.800019 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:00.800036 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:00.801694 (+ 1658us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:09:00.803362 (+ 1668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:09:00.803369 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:09:00.803387 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:09:00.803648 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:09:00.803918 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:09:00.803921 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:09:00.803923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:09:00.804069 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:09:00.804462 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:09:00.804469 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:09:00.804743 (+ 274us) client_negotiation.cc:769] Sending connection context
0811 15:09:00.804925 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 15:09:00.805287 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"thread_start_us":106,"threads_started":1}
I20260811 15:09:00.807796 12067 heartbeater.cc:347] Connected to a master server at 127.30.8.62:34091
WARNING: no policy specified for kudu/127.30.8.2@KRBTEST.COM; defaulting to no policy
I20260811 15:09:00.808037 12067 heartbeater.cc:464] Registering TS with master...
I20260811 15:09:00.808542 12067 heartbeater.cc:511] Master 127.30.8.62:34091 requested a full tablet report, sending...
I20260811 15:09:00.810082 11767 ts_manager.cc:194] Registered new tserver with Master: ede0f9ef8a53435a9702e81726664d4e (127.30.8.1:38981)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.2@KRBTEST.COM" created.
I20260811 15:09:00.811473 11767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:50475
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
I20260811 15:09:00.826570 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:34091
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:40395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:09:00.942631 12074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:09:00.942965 12074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:09:00.943032 12074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:09:00.947136 12074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:09:00.947211 12074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:09:00.947307 12074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:09:00.952872 12074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:40395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:34091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:09:00.954067 12074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:09:00.955191 12074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:09:00.965154 12080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:00.965304 12074 server_base.cc:1034] running on GCE node
W20260811 15:09:00.965169 12082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:00.965174 12079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:00.965811 12074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:09:00.966485 12074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:09:00.967638 12074 hybrid_clock.cc:648] HybridClock initialized: now 1786460940967621 us; error 28 us; skew 500 ppm
Aug 11 15:09:00 dist-test-slave-1vmg krb5kdc[11702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460940, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:09:00.970495 12074 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:09:00.971494 12074 webserver.cc:460] Webserver started at http://127.30.8.2:35373/ using document root <none> and password file <none>
I20260811 15:09:00.972064 12074 fs_manager.cc:359] Metadata directory not provided
I20260811 15:09:00.972127 12074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:09:00.972352 12074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:09:00.974503 12074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "64f9684f20c64b7992cbc6cb89ca9bcc"
format_stamp: "Formatted at 2026-08-11 15:09:00 on dist-test-slave-1vmg"
server_key: "e33f22c496007d2fa2f94e5ca93b66ee"
server_key_iv: "e6ff3fc3811d407ef2e43eb87d9ede5a"
server_key_version: "encryptionkey@0"
I20260811 15:09:00.975090 12074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "64f9684f20c64b7992cbc6cb89ca9bcc"
format_stamp: "Formatted at 2026-08-11 15:09:00 on dist-test-slave-1vmg"
server_key: "e33f22c496007d2fa2f94e5ca93b66ee"
server_key_iv: "e6ff3fc3811d407ef2e43eb87d9ede5a"
server_key_version: "encryptionkey@0"
I20260811 15:09:00.978885 12074 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 15:09:00.981467 12090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:00.982686 12074 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 15:09:00.982796 12074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "64f9684f20c64b7992cbc6cb89ca9bcc"
format_stamp: "Formatted at 2026-08-11 15:09:00 on dist-test-slave-1vmg"
server_key: "e33f22c496007d2fa2f94e5ca93b66ee"
server_key_iv: "e6ff3fc3811d407ef2e43eb87d9ede5a"
server_key_version: "encryptionkey@0"
I20260811 15:09:00.982904 12074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:09:00.999481 12074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:09:01.002601 12074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:09:01.002795 12074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:09:01.003463 12074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:09:01.004356 12074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:09:01.004406 12074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:01.004453 12074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:09:01.004482 12074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:01.013015 12074 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:45761
I20260811 15:09:01.013041 12278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:45761 every 8 connection(s)
I20260811 15:09:01.013953 12074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
Aug 11 15:09:01 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460940, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:09:01.023962 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 12074
I20260811 15:09:01.024088 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:09:01.024362 30752 external_mini_cluster.cc:1428] Setting key c91508eebc2a570588d3647683114cc4
I20260811 15:09:01.027915 11833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:01.015863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:40085 (local address 127.30.8.62:34091)
0811 15:09:01.016004 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:01.016008 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:01.016862 (+ 854us) server_negotiation.cc:407] Connection header received
0811 15:09:01.018153 (+ 1291us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:01.018157 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:01.018218 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:01.018291 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:01.019571 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:01.020121 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:01.021270 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:01.021523 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:01.021611 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:01.024238 (+ 2627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:09:01.024242 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:09:01.024244 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:09:01.024270 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:09:01.025864 (+ 1594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:09:01.026353 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:09:01.026357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:09:01.026358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:09:01.026451 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:09:01.026743 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:09:01.026745 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:09:01.026747 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:09:01.027006 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:09:01.027106 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:01.027612 (+ 506us) server_negotiation.cc:299] Negotiation successful
0811 15:09:01.027750 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 15:09:01.028985 12283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:01.016235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.2:40085)
0811 15:09:01.016699 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:01.016754 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:01.017926 (+ 1172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:01.018477 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:01.018485 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:01.018874 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:01.019425 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:01.019439 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:01.020252 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:01.020255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:01.021147 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:01.021157 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:01.021680 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:01.021684 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:01.021842 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:01.022435 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:01.022453 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:01.024046 (+ 1593us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:09:01.025970 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:09:01.025975 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:09:01.025989 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:09:01.026251 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:09:01.026530 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:09:01.026532 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:09:01.026534 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:09:01.026663 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:09:01.027097 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:09:01.027103 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:09:01.027376 (+ 273us) client_negotiation.cc:769] Sending connection context
0811 15:09:01.027551 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 15:09:01.027870 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":121,"threads_started":1}
I20260811 15:09:01.030160 12279 heartbeater.cc:347] Connected to a master server at 127.30.8.62:34091
I20260811 15:09:01.030421 12279 heartbeater.cc:464] Registering TS with master...
I20260811 15:09:01.030995 12279 heartbeater.cc:511] Master 127.30.8.62:34091 requested a full tablet report, sending...
I20260811 15:09:01.032063 11767 ts_manager.cc:194] Registered new tserver with Master: 64f9684f20c64b7992cbc6cb89ca9bcc (127.30.8.2:45761)
I20260811 15:09:01.032727 11767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:40085
WARNING: no policy specified for kudu/127.30.8.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
I20260811 15:09:01.051846 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:34091
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:40395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:09:01.167869 12287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:09:01.168174 12287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:09:01.168246 12287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:09:01.172327 12287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:09:01.172410 12287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:09:01.172494 12287 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:09:01.178157 12287 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:40395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:34091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12287
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:09:01.179410 12287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:09:01.180500 12287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:09:01.190394 12294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:01.190413 12293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:01.190464 12287 server_base.cc:1034] running on GCE node
W20260811 15:09:01.190419 12298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:01.191085 12287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:09:01.191777 12287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:09:01.192937 12287 hybrid_clock.cc:648] HybridClock initialized: now 1786460941192920 us; error 36 us; skew 500 ppm
Aug 11 15:09:01 dist-test-slave-1vmg krb5kdc[11702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460941, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:09:01.195684 12287 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:09:01.196609 12287 webserver.cc:460] Webserver started at http://127.30.8.3:37771/ using document root <none> and password file <none>
I20260811 15:09:01.197187 12287 fs_manager.cc:359] Metadata directory not provided
I20260811 15:09:01.197257 12287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:09:01.197474 12287 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:09:01.199661 12287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "388b5cf9b4f341d8a982403ba96475a2"
format_stamp: "Formatted at 2026-08-11 15:09:01 on dist-test-slave-1vmg"
server_key: "44cd8128ff715ab4c583ede5b917d961"
server_key_iv: "a86edcfb3b5175e4b65f3c7e7fa2f880"
server_key_version: "encryptionkey@0"
I20260811 15:09:01.200232 12287 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "388b5cf9b4f341d8a982403ba96475a2"
format_stamp: "Formatted at 2026-08-11 15:09:01 on dist-test-slave-1vmg"
server_key: "44cd8128ff715ab4c583ede5b917d961"
server_key_iv: "a86edcfb3b5175e4b65f3c7e7fa2f880"
server_key_version: "encryptionkey@0"
I20260811 15:09:01.204062 12287 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 15:09:01.206521 12310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:01.207747 12287 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 15:09:01.207866 12287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "388b5cf9b4f341d8a982403ba96475a2"
format_stamp: "Formatted at 2026-08-11 15:09:01 on dist-test-slave-1vmg"
server_key: "44cd8128ff715ab4c583ede5b917d961"
server_key_iv: "a86edcfb3b5175e4b65f3c7e7fa2f880"
server_key_version: "encryptionkey@0"
I20260811 15:09:01.207971 12287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:09:01.227500 12287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:09:01.230468 12287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:09:01.230670 12287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:09:01.231401 12287 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:09:01.232309 12287 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:09:01.232360 12287 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:01.232415 12287 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:09:01.232445 12287 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:01.241410 12287 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:42537
I20260811 15:09:01.241432 12488 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:42537 every 8 connection(s)
I20260811 15:09:01.242414 12287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:09:01.248929 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 12287
I20260811 15:09:01.249051 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:09:01.249313 30752 external_mini_cluster.cc:1428] Setting key 6ee7ab02d55b709eefa9c7cf933df34b
Aug 11 15:09:01 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460941, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:09:01.255842 11833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:01.244307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:52637 (local address 127.30.8.62:34091)
0811 15:09:01.244459 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:01.244463 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:01.245307 (+ 844us) server_negotiation.cc:407] Connection header received
0811 15:09:01.246557 (+ 1250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:01.246560 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:01.246608 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:01.246681 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:01.247868 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:01.248397 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:01.249574 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:01.249825 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:01.249926 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:01.252720 (+ 2794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:09:01.252723 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:09:01.252725 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:09:01.252750 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:09:01.253893 (+ 1143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:09:01.254339 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:09:01.254341 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:09:01.254342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:09:01.254412 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:09:01.254669 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:09:01.254671 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:09:01.254672 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:09:01.254863 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:09:01.254999 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:01.255546 (+ 547us) server_negotiation.cc:299] Negotiation successful
0811 15:09:01.255676 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 15:09:01.256706 12493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:01.244706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.3:52637)
0811 15:09:01.245157 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:01.245200 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:01.246361 (+ 1161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:01.246840 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:01.246846 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:01.247236 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:01.247747 (+ 511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:01.247761 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:01.248489 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:01.248491 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:01.249456 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:01.249467 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:01.249969 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:01.249973 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:01.250182 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:01.250827 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:01.250850 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:01.252557 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:09:01.253991 (+ 1434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:09:01.253996 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:09:01.254010 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:09:01.254263 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:09:01.254473 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:09:01.254475 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:09:01.254476 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:09:01.254606 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:09:01.254997 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:09:01.255005 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:09:01.255302 (+ 297us) client_negotiation.cc:769] Sending connection context
0811 15:09:01.255488 (+ 186us) client_negotiation.cc:240] Negotiation successful
0811 15:09:01.255850 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":114,"threads_started":1}
I20260811 15:09:01.257827 12489 heartbeater.cc:347] Connected to a master server at 127.30.8.62:34091
I20260811 15:09:01.258077 12489 heartbeater.cc:464] Registering TS with master...
I20260811 15:09:01.258589 12489 heartbeater.cc:511] Master 127.30.8.62:34091 requested a full tablet report, sending...
I20260811 15:09:01.259665 11767 ts_manager.cc:194] Registered new tserver with Master: 388b5cf9b4f341d8a982403ba96475a2 (127.30.8.3:42537)
I20260811 15:09:01.260241 11767 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:52637
I20260811 15:09:01.263459 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.1.keytab.
Entry for principal oryx/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.1.keytab.
WARNING: no policy specified for oryx/127.30.8.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.8.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.2.keytab.
Entry for principal oryx/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.2.keytab.
WARNING: no policy specified for oryx/127.30.8.3@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.8.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.3.keytab.
Entry for principal oryx/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.3.keytab.
I20260811 15:09:01.343880 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 11865
I20260811 15:09:01.372512 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 12074
I20260811 15:09:01.385073 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 12287
I20260811 15:09:01.408944 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 11718
I20260811 15:09:01.424181 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:34091
--webserver_interface=127.30.8.62
--webserver_port=44009
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:40395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:34091
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:09:01.538887 12505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:09:01.539201 12505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:09:01.539242 12505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:09:01.543346 12505 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:09:01.543421 12505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:09:01.543439 12505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:09:01.543455 12505 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:09:01.543476 12505 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:09:01.548949 12505 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:40395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:34091
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:34091
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=44009
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:09:01.550144 12505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:09:01.551219 12505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:09:01.559702 12511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:01.559707 12512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:01.559724 12514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:01.559908 12505 server_base.cc:1034] running on GCE node
I20260811 15:09:01.560562 12505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:09:01.561733 12505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:09:01.562896 12505 hybrid_clock.cc:648] HybridClock initialized: now 1786460941562880 us; error 32 us; skew 500 ppm
Aug 11 15:09:01 dist-test-slave-1vmg krb5kdc[11702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460941, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:09:01.565768 12505 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:09:01.566725 12505 webserver.cc:460] Webserver started at http://127.30.8.62:44009/ using document root <none> and password file <none>
I20260811 15:09:01.567334 12505 fs_manager.cc:359] Metadata directory not provided
I20260811 15:09:01.567425 12505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:09:01.571439 12505 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260811 15:09:01.573566 12527 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:01.574803 12505 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 15:09:01.574965 12505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "bfc279b5adcf4ad7a07020f1efb590b5"
format_stamp: "Formatted at 2026-08-11 15:09:00 on dist-test-slave-1vmg"
server_key: "df9937669317a92b1c19b57922a7cfe6"
server_key_iv: "53b9763cb5c0523d6f1def0cef3920fb"
server_key_version: "encryptionkey@0"
I20260811 15:09:01.575546 12505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:09:01.586735 12505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:09:01.589635 12505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:09:01.589841 12505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:09:01.596946 12505 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:34091
I20260811 15:09:01.596983 12616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:34091 every 8 connection(s)
I20260811 15:09:01.598054 12505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:09:01.600517 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 12505
I20260811 15:09:01.600931 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:38981
--local_ip_for_outbound_sockets=127.30.8.1
--tserver_master_addrs=127.30.8.62:34091
--webserver_port=43677
--webserver_interface=127.30.8.1
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:40395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.8.1
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.1.keytab with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:09:01.602741 12617 sys_catalog.cc:263] Verifying existing consensus state
I20260811 15:09:01.605019 12617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5: Bootstrap starting.
I20260811 15:09:01.627998 12617 log.cc:824] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5: Log is configured to *not* fsync() on all Append() calls
I20260811 15:09:01.636813 12617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 15:09:01.637307 12617 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5: Bootstrap complete.
I20260811 15:09:01.643445 12617 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc279b5adcf4ad7a07020f1efb590b5" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34091 } }
I20260811 15:09:01.644052 12617 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfc279b5adcf4ad7a07020f1efb590b5, State: Initialized, Role: FOLLOWER
I20260811 15:09:01.644665 12617 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc279b5adcf4ad7a07020f1efb590b5" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34091 } }
I20260811 15:09:01.644829 12617 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:09:01.644910 12617 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:09:01.645027 12617 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [term 1 FOLLOWER]: Advancing to term 2
I20260811 15:09:01.647929 12617 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc279b5adcf4ad7a07020f1efb590b5" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34091 } }
I20260811 15:09:01.648360 12617 leader_election.cc:302] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfc279b5adcf4ad7a07020f1efb590b5; no voters:
I20260811 15:09:01.648800 12617 leader_election.cc:288] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 15:09:01.648919 12623 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 15:09:01.649183 12623 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [term 2 LEADER]: Becoming Leader. State: Replica: bfc279b5adcf4ad7a07020f1efb590b5, State: Running, Role: LEADER
I20260811 15:09:01.649559 12623 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc279b5adcf4ad7a07020f1efb590b5" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34091 } }
I20260811 15:09:01.650269 12617 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:09:01.651000 12625 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfc279b5adcf4ad7a07020f1efb590b5. Latest consensus state: current_term: 2 leader_uuid: "bfc279b5adcf4ad7a07020f1efb590b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc279b5adcf4ad7a07020f1efb590b5" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34091 } } }
I20260811 15:09:01.651147 12625 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [sys.catalog]: This master's current role is: LEADER
I20260811 15:09:01.651014 12624 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bfc279b5adcf4ad7a07020f1efb590b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfc279b5adcf4ad7a07020f1efb590b5" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34091 } } }
I20260811 15:09:01.651218 12624 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5 [sys.catalog]: This master's current role is: LEADER
I20260811 15:09:01.651520 12632 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:09:01.654956 12632 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:09:01.655735 12632 catalog_manager.cc:1259] Loaded cluster ID: f9702c74f96a496484c9edaec2d94166
I20260811 15:09:01.655788 12632 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:09:01.657560 12632 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:09:01.658125 12632 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bfc279b5adcf4ad7a07020f1efb590b5: Loaded TSK: 0
I20260811 15:09:01.659094 12632 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 15:09:01.765450 12619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:09:01.765774 12619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:09:01.765841 12619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:09:01.770135 12619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:09:01.770222 12619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:09:01.770308 12619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:09:01.776010 12619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:40395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.1.keytab
--principal=oryx/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:38981
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=43677
--tserver_master_addrs=127.30.8.62:34091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12619
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:09:01.777122 12619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:09:01.778119 12619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:09:01.787916 12653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:01.788064 12654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:01.787909 12657 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:01.788419 12619 server_base.cc:1034] running on GCE node
I20260811 15:09:01.788923 12619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:09:01.789762 12619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:09:01.790983 12619 hybrid_clock.cc:648] HybridClock initialized: now 1786460941790956 us; error 40 us; skew 500 ppm
Aug 11 15:09:01 dist-test-slave-1vmg krb5kdc[11702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460941, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:09:01.794641 12619 init.cc:377] Logged in from keytab as oryx/127.30.8.1@KRBTEST.COM (short username oryx)
I20260811 15:09:01.795938 12619 webserver.cc:460] Webserver started at http://127.30.8.1:43677/ using document root <none> and password file <none>
I20260811 15:09:01.796685 12619 fs_manager.cc:359] Metadata directory not provided
I20260811 15:09:01.796804 12619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:09:01.802223 12619 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 15:09:01.805054 12666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:01.806617 12619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:09:01.806782 12619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "ede0f9ef8a53435a9702e81726664d4e"
format_stamp: "Formatted at 2026-08-11 15:09:00 on dist-test-slave-1vmg"
server_key: "27aa328d957d9d00cf63f0a72a77c888"
server_key_iv: "739446d72d1ca91f421fd1c1c700ad7a"
server_key_version: "encryptionkey@0"
I20260811 15:09:01.807585 12619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:09:01.828538 12619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:09:01.832448 12619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:09:01.832695 12619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:09:01.833503 12619 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:09:01.834664 12619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:09:01.834740 12619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:01.834815 12619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:09:01.834856 12619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:01.847344 12619 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:38981
I20260811 15:09:01.847357 12848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:38981 every 8 connection(s)
I20260811 15:09:01.848719 12619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:09:01.849649 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 12619
I20260811 15:09:01.850126 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:45761
--local_ip_for_outbound_sockets=127.30.8.2
--tserver_master_addrs=127.30.8.62:34091
--webserver_port=35373
--webserver_interface=127.30.8.2
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:40395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.8.2
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.2.keytab with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 15:09:01 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:01 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:01.864354 12858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:01.851558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.1:60791)
0811 15:09:01.852189 (+ 631us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:01.852255 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:01.853800 (+ 1545us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:01.855068 (+ 1268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:01.855078 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:01.855580 (+ 502us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:01.856325 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:01.856349 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:01.857655 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:01.857659 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:01.859094 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:01.859110 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:01.859905 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:01.859911 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:01.860142 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:01.860815 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:01.860841 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:01.863522 (+ 2681us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":370,"thread_start_us":161,"threads_started":1}
W20260811 15:09:01.864758 12863 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:01.852864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:60791 (local address 127.30.8.62:34091)
0811 15:09:01.853462 (+ 598us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:01.853479 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:01.853526 (+ 47us) server_negotiation.cc:407] Connection header received
0811 15:09:01.854247 (+ 721us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:01.854259 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:01.854575 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:01.854856 (+ 281us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:01.856524 (+ 1668us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:01.857497 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:01.859230 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:01.859702 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:01.859849 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:01.863855 (+ 4006us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.1:60791: BlockingRecv error: recv got EOF from 127.30.8.1:60791 (error 108)
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":113,"threads_started":1}
W20260811 15:09:01.864952 12854 heartbeater.cc:644] Failed to heartbeat to 127.30.8.62:34091 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.8.62:34091: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Aug 11 15:09:01 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:01 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:01.870988 12858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:01.865563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.1:39563)
0811 15:09:01.865751 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:01.865763 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:01.865832 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:01.866089 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:01.866091 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:01.866250 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:01.866363 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:01.866369 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:01.867466 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:01.867468 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:01.868023 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:01.868030 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:01.868602 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:01.868604 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:01.868669 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:01.869138 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:01.869150 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:01.870847 (+ 1697us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":108}
W20260811 15:09:01.871213 12863 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:01.865671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:39563 (local address 127.30.8.62:34091)
0811 15:09:01.865791 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:01.865796 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:01.865814 (+ 18us) server_negotiation.cc:407] Connection header received
0811 15:09:01.865919 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:01.865923 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:01.865978 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:01.866085 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:01.866471 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:01.867357 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:01.868128 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:01.868497 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:01.868611 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:01.871079 (+ 2468us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.1:39563: BlockingRecv error: recv got EOF from 127.30.8.1:39563 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 11 15:09:01 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:01 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:01.876322 12858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:01.871519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.1:41663)
0811 15:09:01.871797 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:01.871812 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:01.871900 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:01.872129 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:01.872131 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:01.872287 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:01.872364 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:01.872369 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:01.873071 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:01.873074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:01.873588 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:01.873596 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:01.874013 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:01.874017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:01.874090 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:01.874548 (+ 458us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:01.874560 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:01.876168 (+ 1608us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":121,"threads_started":1}
W20260811 15:09:01.876420 12863 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:01.871628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:41663 (local address 127.30.8.62:34091)
0811 15:09:01.871748 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:01.871752 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:01.871874 (+ 122us) server_negotiation.cc:407] Connection header received
0811 15:09:01.871993 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:01.871996 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:01.872043 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:01.872102 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:01.872469 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:01.872971 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:01.873696 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:01.873917 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:01.874003 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:01.876314 (+ 2311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.1:41663: BlockingRecv error: recv got EOF from 127.30.8.1:41663 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 15:09:01.876694 12854 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 15:09:01.973984 12861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:09:01.974299 12861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:09:01.974347 12861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:09:01.978528 12861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:09:01.978616 12861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:09:01.978713 12861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:09:01.984368 12861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:40395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.2.keytab
--principal=oryx/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:45761
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=35373
--tserver_master_addrs=127.30.8.62:34091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:09:01.985518 12861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:09:01.986573 12861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:09:01.996641 12871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:01.996650 12873 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:01.997025 12870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:01.997308 12861 server_base.cc:1034] running on GCE node
I20260811 15:09:01.997812 12861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:09:01.998585 12861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:09:01.999770 12861 hybrid_clock.cc:648] HybridClock initialized: now 1786460941999741 us; error 51 us; skew 500 ppm
Aug 11 15:09:01 dist-test-slave-1vmg krb5kdc[11702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460941, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:09:02.002518 12861 init.cc:377] Logged in from keytab as oryx/127.30.8.2@KRBTEST.COM (short username oryx)
I20260811 15:09:02.003454 12861 webserver.cc:460] Webserver started at http://127.30.8.2:35373/ using document root <none> and password file <none>
I20260811 15:09:02.003994 12861 fs_manager.cc:359] Metadata directory not provided
I20260811 15:09:02.004068 12861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:09:02.007921 12861 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 15:09:02.009936 12882 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:02.011188 12861 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 15:09:02.011303 12861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "64f9684f20c64b7992cbc6cb89ca9bcc"
format_stamp: "Formatted at 2026-08-11 15:09:00 on dist-test-slave-1vmg"
server_key: "e33f22c496007d2fa2f94e5ca93b66ee"
server_key_iv: "e6ff3fc3811d407ef2e43eb87d9ede5a"
server_key_version: "encryptionkey@0"
I20260811 15:09:02.011854 12861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:09:02.026979 12861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:09:02.030040 12861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:09:02.030220 12861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:09:02.030812 12861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:09:02.031659 12861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:09:02.031708 12861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:02.031754 12861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:09:02.031785 12861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:02.040050 12861 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:45761
I20260811 15:09:02.040129 13049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:45761 every 8 connection(s)
I20260811 15:09:02.040971 12861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
Aug 11 15:09:02 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:02.049659 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 12861
Aug 11 15:09:02 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:02.050166 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:42537
--local_ip_for_outbound_sockets=127.30.8.3
--tserver_master_addrs=127.30.8.62:34091
--webserver_port=37771
--webserver_interface=127.30.8.3
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:40395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.8.3
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.3.keytab with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:09:02.051256 12863 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:02.042615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:33217 (local address 127.30.8.62:34091)
0811 15:09:02.042738 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:02.042741 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:02.043526 (+ 785us) server_negotiation.cc:407] Connection header received
0811 15:09:02.044763 (+ 1237us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:02.044767 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:02.044832 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:02.044914 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:02.046040 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.046566 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.047723 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.047943 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.048054 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:02.051118 (+ 3064us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.2:33217: BlockingRecv error: recv got EOF from 127.30.8.2:33217 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 15:09:02.051792 13052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:02.042901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.2:33217)
0811 15:09:02.043371 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:02.043412 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:02.044551 (+ 1139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:02.045092 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:02.045099 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:02.045427 (+ 328us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:02.045880 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:02.045894 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.046696 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.046699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:02.047591 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:02.047603 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.048088 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.048092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:02.048256 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:02.048702 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:02.048719 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:02.050961 (+ 2242us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":110,"threads_started":1}
W20260811 15:09:02.052389 13050 heartbeater.cc:644] Failed to heartbeat to 127.30.8.62:34091 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.8.62:34091: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Aug 11 15:09:02 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:02 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:02.057888 13052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:02.052811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.2:42875)
0811 15:09:02.052934 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:02.052949 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:02.053044 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:02.053264 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:02.053267 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:02.053430 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:02.053548 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:02.053556 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.054443 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.054445 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:02.055035 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:02.055045 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.055579 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.055582 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:02.055652 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:02.056076 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:02.056090 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:02.057805 (+ 1715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 15:09:02.058071 12863 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:02.052914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:42875 (local address 127.30.8.62:34091)
0811 15:09:02.053013 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:02.053017 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:02.053030 (+ 13us) server_negotiation.cc:407] Connection header received
0811 15:09:02.053135 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:02.053137 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:02.053168 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:02.053231 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:02.053676 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.054347 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.055147 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.055473 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.055580 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:02.057960 (+ 2380us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.2:42875: BlockingRecv error: recv got EOF from 127.30.8.2:42875 (error 108)
Metrics: {"server-negotiator.queue_time_us":29}
Aug 11 15:09:02 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:02 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:02.063369 13052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:02.058429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.2:38755)
0811 15:09:02.058649 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:02.058661 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:02.058733 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:02.058977 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:02.058979 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:02.059102 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:02.059169 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:02.059174 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.060070 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.060072 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:02.060562 (+ 490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:02.060569 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.061075 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.061078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:02.061131 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:02.061524 (+ 393us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:02.061537 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:02.063268 (+ 1731us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":161}
W20260811 15:09:02.063589 12863 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:02.058541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:38755 (local address 127.30.8.62:34091)
0811 15:09:02.058656 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:02.058660 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:02.058704 (+ 44us) server_negotiation.cc:407] Connection header received
0811 15:09:02.058807 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:02.058810 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:02.058852 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:02.058961 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:02.059271 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.059976 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.060668 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.060976 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.061084 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:02.063398 (+ 2314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.2:38755: BlockingRecv error: recv got EOF from 127.30.8.2:38755 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
W20260811 15:09:02.063584 13050 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 15:09:02.160100 13054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:09:02.160303 13054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:09:02.160341 13054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:09:02.164537 13054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:09:02.164584 13054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:09:02.164638 13054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:09:02.169556 13054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:40395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/krb5kdc/oryx_127.30.8.3.keytab
--principal=oryx/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:42537
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=37771
--tserver_master_addrs=127.30.8.62:34091
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:09:02.170353 13054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:09:02.171217 13054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:09:02.179976 13066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:02.180013 13062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:02.179977 13063 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:02.180197 13054 server_base.cc:1034] running on GCE node
I20260811 15:09:02.180552 13054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:09:02.181103 13054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:09:02.182263 13054 hybrid_clock.cc:648] HybridClock initialized: now 1786460942182237 us; error 37 us; skew 500 ppm
Aug 11 15:09:02 dist-test-slave-1vmg krb5kdc[11702](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460942, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:09:02.184846 13054 init.cc:377] Logged in from keytab as oryx/127.30.8.3@KRBTEST.COM (short username oryx)
I20260811 15:09:02.185587 13054 webserver.cc:460] Webserver started at http://127.30.8.3:37771/ using document root <none> and password file <none>
I20260811 15:09:02.186067 13054 fs_manager.cc:359] Metadata directory not provided
I20260811 15:09:02.186144 13054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:09:02.189626 13054 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.002s sys 0.000s
I20260811 15:09:02.191468 13076 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:02.192585 13054 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 15:09:02.192694 13054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "388b5cf9b4f341d8a982403ba96475a2"
format_stamp: "Formatted at 2026-08-11 15:09:01 on dist-test-slave-1vmg"
server_key: "44cd8128ff715ab4c583ede5b917d961"
server_key_iv: "a86edcfb3b5175e4b65f3c7e7fa2f880"
server_key_version: "encryptionkey@0"
I20260811 15:09:02.193209 13054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:09:02.208618 13054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:09:02.211001 13054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:09:02.211138 13054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:09:02.211602 13054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:09:02.212322 13054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:09:02.212369 13054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:02.212402 13054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:09:02.212422 13054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:02.220824 13054 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:42537
I20260811 15:09:02.220857 13248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:42537 every 8 connection(s)
I20260811 15:09:02.221673 13054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:09:02.227584 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 13054
Aug 11 15:09:02 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:02 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
W20260811 15:09:02.231871 12863 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:02.223251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:45955 (local address 127.30.8.62:34091)
0811 15:09:02.223383 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:02.223388 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:02.224151 (+ 763us) server_negotiation.cc:407] Connection header received
0811 15:09:02.225260 (+ 1109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:02.225263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:02.225308 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:02.225378 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:02.226372 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.226866 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.227979 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.228198 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.228303 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:02.231746 (+ 3443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.3:45955: BlockingRecv error: recv got EOF from 127.30.8.3:45955 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 15:09:02.232389 13256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:02.223594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.3:45955)
0811 15:09:02.224012 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:02.224050 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:02.225080 (+ 1030us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:02.225517 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:02.225523 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:02.225841 (+ 318us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:02.226263 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:02.226276 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.226979 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.226982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:02.227867 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:02.227877 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.228350 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.228354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:02.228542 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:02.229116 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:02.229138 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:02.231612 (+ 2474us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":120,"threads_started":1}
W20260811 15:09:02.232859 13253 heartbeater.cc:644] Failed to heartbeat to 127.30.8.62:34091 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.8.62:34091: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
I20260811 15:09:02.236837 13259 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:02.228495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43234 (local address 127.30.8.62:34091)
0811 15:09:02.228697 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:02.228700 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:02.228711 (+ 11us) server_negotiation.cc:407] Connection header received
0811 15:09:02.228741 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:02.228743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:02.228778 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:02.228871 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:02.229703 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.230252 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.231168 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.231499 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.231614 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:02.232996 (+ 1382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:09:02.233008 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:09:02.233025 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:09:02.233056 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:09:02.234729 (+ 1673us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:09:02.235194 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:09:02.235200 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:09:02.235206 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:09:02.235363 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:09:02.235618 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:09:02.235620 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:09:02.235622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:09:02.235994 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:09:02.236168 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:02.236464 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 15:09:02.236682 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":64,"threads_started":1}
Aug 11 15:09:02 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:02 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:02.238256 13256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:02.233223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.3:60609)
0811 15:09:02.233365 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:02.233378 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:02.233440 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:02.233649 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:02.233652 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:02.233807 (+ 155us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:02.233912 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:02.233919 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.234605 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.234609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:02.235374 (+ 765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:02.235383 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.235861 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.235864 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:02.235955 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:02.236378 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:02.236392 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:02.238166 (+ 1774us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":86}
W20260811 15:09:02.238392 12863 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:02.233315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:60609 (local address 127.30.8.62:34091)
0811 15:09:02.233451 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:02.233455 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:02.233471 (+ 16us) server_negotiation.cc:407] Connection header received
0811 15:09:02.233505 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:02.233507 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:02.233547 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:02.233620 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:02.234020 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.234495 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.235490 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.235756 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.235834 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:02.238293 (+ 2459us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.3:60609: BlockingRecv error: recv got EOF from 127.30.8.3:60609 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":61}
Aug 11 15:09:02 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:02 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:02.242635 13256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:02.238664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.3:57525)
0811 15:09:02.238759 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:02.238771 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:02.238834 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:02.239087 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:02.239090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:02.239243 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:02.239320 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:02.239326 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.240013 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.240015 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:02.240496 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:02.240504 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.240841 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.240843 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:02.240894 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:02.241204 (+ 310us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:02.241221 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:02.242560 (+ 1339us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":25}
W20260811 15:09:02.242769 12863 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:02.238743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:57525 (local address 127.30.8.62:34091)
0811 15:09:02.238857 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:02.238861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:02.238873 (+ 12us) server_negotiation.cc:407] Connection header received
0811 15:09:02.238905 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:02.238907 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:02.238978 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:02.239053 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:02.239408 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.239922 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.240580 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.240771 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.240835 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:02.242667 (+ 1832us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.3:57525: BlockingRecv error: recv got EOF from 127.30.8.3:57525 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 15:09:02.242800 13253 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 15:09:02 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:02 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:02.882490 13262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:02.877249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.1:50929)
0811 15:09:02.877501 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:02.877513 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:02.877590 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:02.877842 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:02.877843 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:02.878006 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:02.878118 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:02.878123 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.878953 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.878956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:02.879519 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:02.879527 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.880022 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.880026 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:02.880109 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:02.880587 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:02.880599 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:02.882367 (+ 1768us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":92,"threads_started":1}
W20260811 15:09:02.882628 13263 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:02.877323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:50929 (local address 127.30.8.62:34091)
0811 15:09:02.877592 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:02.877595 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:02.877610 (+ 15us) server_negotiation.cc:407] Connection header received
0811 15:09:02.877693 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:02.877695 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:02.877739 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:02.877816 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:02.878203 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.878812 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.879618 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:02.879894 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:02.879999 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:02.882492 (+ 2493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.1:50929: BlockingRecv error: recv got EOF from 127.30.8.1:50929 (error 108)
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":92,"threads_started":1}
Aug 11 15:09:03 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:03 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:03.070163 13265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:03.064231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.2:48339)
0811 15:09:03.064581 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:03.064595 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:03.064721 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:03.065013 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:03.065016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:03.065234 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:03.065336 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:03.065343 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:03.066127 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:03.066131 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:03.066736 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:03.066747 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:03.067278 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:03.067283 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:03.067371 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:03.067925 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:03.067938 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:03.069969 (+ 2031us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":117,"threads_started":1}
W20260811 15:09:03.070286 13263 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:03.064309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:48339 (local address 127.30.8.62:34091)
0811 15:09:03.064508 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:03.064513 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:03.064674 (+ 161us) server_negotiation.cc:407] Connection header received
0811 15:09:03.064822 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:03.064826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:03.064879 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:03.064961 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:03.065462 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:03.066008 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:03.066863 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:03.067164 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:03.067269 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:03.070168 (+ 2899us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.2:48339: BlockingRecv error: recv got EOF from 127.30.8.2:48339 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 11 15:09:03 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:03 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:03.248620 13266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:03.243507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.3:57955)
0811 15:09:03.243777 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:03.243792 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:03.243891 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:03.244088 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:03.244090 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:03.244279 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:03.244379 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:03.244384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:03.245066 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:03.245069 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:03.245644 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:03.245652 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:03.246019 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:03.246021 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:03.246095 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:03.246572 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:03.246585 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:03.248484 (+ 1899us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":89,"threads_started":1}
W20260811 15:09:03.248731 13263 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:03.243534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:57955 (local address 127.30.8.62:34091)
0811 15:09:03.243661 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:03.243665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:03.243850 (+ 185us) server_negotiation.cc:407] Connection header received
0811 15:09:03.243968 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:03.243970 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:03.244016 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:03.244071 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:03.244461 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:03.244978 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:03.245731 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:03.245939 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:03.246008 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:03.248627 (+ 2619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.3:57955: BlockingRecv error: recv got EOF from 127.30.8.3:57955 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 15:09:03 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:03 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:03.888948 13269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:03.883383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.1:41961)
0811 15:09:03.883707 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:03.883721 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:03.883824 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:03.884078 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:03.884081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:03.884268 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:03.884374 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:03.884380 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:03.885096 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:03.885099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:03.885647 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:03.885654 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:03.886086 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:03.886088 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:03.886147 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:03.886631 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:03.886642 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:03.888802 (+ 2160us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":232,"thread_start_us":101,"threads_started":1}
W20260811 15:09:03.889031 13272 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:03.883496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:41961 (local address 127.30.8.62:34091)
0811 15:09:03.883779 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:03.883783 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:03.883798 (+ 15us) server_negotiation.cc:407] Connection header received
0811 15:09:03.883914 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:03.883916 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:03.883960 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:03.884040 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:03.884464 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:03.885006 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:03.885759 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:03.885993 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:03.886071 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:03.888921 (+ 2850us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.1:41961: BlockingRecv error: recv got EOF from 127.30.8.1:41961 (error 108)
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":74,"threads_started":1}
Aug 11 15:09:04 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:04 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:04.077096 13273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:04.071339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.2:49653)
0811 15:09:04.071659 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:04.071675 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:04.071807 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:04.072076 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:04.072079 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:04.072297 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:04.072418 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:04.072426 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:04.073421 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:04.073424 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:04.074018 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:04.074029 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:04.074486 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:04.074489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:04.074570 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:04.075114 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:04.075126 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:04.076941 (+ 1815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":101,"threads_started":1}
W20260811 15:09:04.077203 13272 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:04.071502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:49653 (local address 127.30.8.62:34091)
0811 15:09:04.071654 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:04.071659 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:04.071752 (+ 93us) server_negotiation.cc:407] Connection header received
0811 15:09:04.071907 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:04.071910 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:04.071960 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:04.072032 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:04.072578 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:04.073275 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:04.074153 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:04.074378 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:04.074468 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:04.077090 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.2:49653: BlockingRecv error: recv got EOF from 127.30.8.2:49653 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 15:09:04 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:04 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:04.254761 13275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:04.249440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.3:32831)
0811 15:09:04.249734 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:04.249747 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:04.249868 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:04.250127 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:04.250129 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:04.250321 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:04.250423 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:04.250428 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:04.251210 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:04.251212 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:04.251826 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:04.251836 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:04.252249 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:04.252252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:04.252310 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:04.252830 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:04.252843 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:04.254596 (+ 1753us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":89,"threads_started":1}
W20260811 15:09:04.254856 13272 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:04.249643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:32831 (local address 127.30.8.62:34091)
0811 15:09:04.249765 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:04.249768 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:04.249800 (+ 32us) server_negotiation.cc:407] Connection header received
0811 15:09:04.249958 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:04.249961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:04.250009 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:04.250088 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:04.250511 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:04.251090 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:04.251916 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:04.252156 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:04.252238 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:04.254743 (+ 2505us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.3:32831: BlockingRecv error: recv got EOF from 127.30.8.3:32831 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 15:09:04 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:04 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:04.895570 13276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:04.889856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.1:59439)
0811 15:09:04.890187 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:04.890201 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:04.890298 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:04.890533 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:04.890535 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:04.890725 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:04.890827 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:04.890832 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:04.891553 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:04.891555 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:04.892157 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:04.892166 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:04.892690 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:04.892693 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:04.892762 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:04.893240 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:04.893251 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:04.895434 (+ 2183us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":104,"threads_started":1}
W20260811 15:09:04.895669 13277 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:04.890040 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:59439 (local address 127.30.8.62:34091)
0811 15:09:04.890309 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:04.890313 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:04.890348 (+ 35us) server_negotiation.cc:407] Connection header received
0811 15:09:04.890388 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:04.890391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:04.890435 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:04.890504 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:04.890936 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:04.891466 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:04.892272 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:04.892584 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:04.892666 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:04.895546 (+ 2880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.1:59439: BlockingRecv error: recv got EOF from 127.30.8.1:59439 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":90,"threads_started":1}
Aug 11 15:09:05 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:05 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:05.083462 13278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:05.077956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.2:58541)
0811 15:09:05.078282 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:05.078296 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:05.078408 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:05.078668 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:05.078670 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:05.078856 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:05.079009 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:05.079016 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:05.079782 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:05.079785 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:05.080366 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:05.080376 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:05.080797 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:05.080799 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:05.080859 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:05.081407 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:05.081420 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:05.083314 (+ 1894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":86,"threads_started":1}
W20260811 15:09:05.083549 13277 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:05.078094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:58541 (local address 127.30.8.62:34091)
0811 15:09:05.078262 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:05.078266 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:05.078363 (+ 97us) server_negotiation.cc:407] Connection header received
0811 15:09:05.078485 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:05.078488 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:05.078534 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:05.078603 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:05.079107 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:05.079672 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:05.080472 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:05.080708 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:05.080784 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:05.083450 (+ 2666us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.2:58541: BlockingRecv error: recv got EOF from 127.30.8.2:58541 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 15:09:05 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:05 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:05.261245 13280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:05.255620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.3:54973)
0811 15:09:05.255932 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:05.255946 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:05.256071 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:05.256343 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:05.256346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:05.256561 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:05.256664 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:05.256671 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:05.257455 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:05.257459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:05.258098 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:05.258109 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:05.258645 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:05.258649 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:05.258736 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:05.259277 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:05.259289 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:05.261037 (+ 1748us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":124,"threads_started":1}
W20260811 15:09:05.261305 13277 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:05.255757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:54973 (local address 127.30.8.62:34091)
0811 15:09:05.255908 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:05.255913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:05.256021 (+ 108us) server_negotiation.cc:407] Connection header received
0811 15:09:05.256181 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:05.256184 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:05.256232 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:05.256297 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:05.256792 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:05.257341 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:05.258230 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:05.258532 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:05.258627 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:05.261202 (+ 2575us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.3:54973: BlockingRecv error: recv got EOF from 127.30.8.3:54973 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 15:09:05 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:05 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:05.903293 13281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:05.896406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.1:34097)
0811 15:09:05.896770 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:05.896786 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:05.896922 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:05.897293 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:05.897297 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:05.897543 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:05.897690 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:05.897697 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:05.898701 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:05.898705 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:05.899461 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:05.899476 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:05.900123 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:05.900128 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:05.900242 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:05.900830 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:05.900843 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:05.903101 (+ 2258us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":104,"threads_started":1}
W20260811 15:09:05.903440 13282 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:05.896512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:34097 (local address 127.30.8.62:34091)
0811 15:09:05.896843 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:05.896847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:05.896864 (+ 17us) server_negotiation.cc:407] Connection header received
0811 15:09:05.897044 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:05.897046 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:05.897105 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:05.897215 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:05.897804 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:05.898356 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:05.899607 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:05.899885 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:05.899981 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:05.903312 (+ 3331us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.1:34097: BlockingRecv error: recv got EOF from 127.30.8.1:34097 (error 108)
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":69,"threads_started":1}
Aug 11 15:09:06 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:06 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:06.091434 13287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:06.084437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.2:46109)
0811 15:09:06.084874 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:06.084895 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:06.085101 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:06.085473 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:06.085476 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:06.085738 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:06.085891 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:06.085899 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:06.086745 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:06.086749 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:06.087638 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:06.087655 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:06.088359 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:06.088366 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:06.088494 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:06.089217 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:06.089230 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:06.091260 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":96,"threads_started":1}
W20260811 15:09:06.091578 13282 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:06.084738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:46109 (local address 127.30.8.62:34091)
0811 15:09:06.084903 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:06.084907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:06.085014 (+ 107us) server_negotiation.cc:407] Connection header received
0811 15:09:06.085211 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:06.085214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:06.085269 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:06.085353 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:06.086022 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:06.086572 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:06.087812 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:06.088155 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:06.088284 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:06.091465 (+ 3181us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.2:46109: BlockingRecv error: recv got EOF from 127.30.8.2:46109 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 11 15:09:06 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:06 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:06.269519 13288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:06.262373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.3:43817)
0811 15:09:06.262873 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:06.262890 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:06.263060 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:06.263353 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:06.263355 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:06.263599 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:06.263757 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:06.263763 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:06.264530 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:06.264533 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:06.265168 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:06.265177 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:06.265720 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:06.265724 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:06.265822 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:06.266513 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:06.266528 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:06.269361 (+ 2833us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":401,"thread_start_us":140,"threads_started":1}
W20260811 15:09:06.269685 13282 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:06.262389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:43817 (local address 127.30.8.62:34091)
0811 15:09:06.262547 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:06.262552 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:06.263006 (+ 454us) server_negotiation.cc:407] Connection header received
0811 15:09:06.263162 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:06.263164 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:06.263224 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:06.263310 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:06.263860 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:06.264413 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:06.265272 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:06.265499 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:06.265716 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:06.269574 (+ 3858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.3:43817: BlockingRecv error: recv got EOF from 127.30.8.3:43817 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 15:09:06 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:06 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:06.911379 13290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:06.904584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.1:50549)
0811 15:09:06.904992 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:06.905009 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:06.905161 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:06.905574 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:06.905578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:06.905924 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:06.906066 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:06.906080 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:06.907223 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:06.907227 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:06.907844 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:06.907855 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:06.908489 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:06.908493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:06.908576 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:06.909175 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:06.909191 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:06.911244 (+ 2053us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":128,"threads_started":1}
W20260811 15:09:06.911593 13291 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:06.904732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:50549 (local address 127.30.8.62:34091)
0811 15:09:06.905018 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:06.905022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:06.905105 (+ 83us) server_negotiation.cc:407] Connection header received
0811 15:09:06.905265 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:06.905268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:06.905319 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:06.905436 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:06.906317 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:06.907068 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:06.908056 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:06.908353 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:06.908451 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:06.911428 (+ 2977us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.1:50549: BlockingRecv error: recv got EOF from 127.30.8.1:50549 (error 108)
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":98,"threads_started":1}
Aug 11 15:09:07 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:07 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:07.097379 13295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:07.092331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.2:39313)
0811 15:09:07.092612 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:07.092625 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:07.092725 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:07.092939 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:07.092941 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:07.093133 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:07.093243 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:07.093249 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:07.093940 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:07.093942 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:07.094521 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:07.094528 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:07.094886 (+ 358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:07.094889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:07.095001 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:07.095464 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:07.095475 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:07.097255 (+ 1780us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":86,"threads_started":1}
W20260811 15:09:07.097566 13291 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:07.092469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:39313 (local address 127.30.8.62:34091)
0811 15:09:07.092626 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:07.092630 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:07.092687 (+ 57us) server_negotiation.cc:407] Connection header received
0811 15:09:07.092798 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:07.092800 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:07.092844 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:07.092910 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:07.093329 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:07.093863 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:07.094602 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:07.094808 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:07.094873 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:07.097394 (+ 2521us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.2:39313: BlockingRecv error: recv got EOF from 127.30.8.2:39313 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 15:09:07 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:07 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:07.275763 13297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:07.270599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.3:55491)
0811 15:09:07.270889 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:07.270903 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:07.271028 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:07.271246 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:07.271248 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:07.271440 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:07.271550 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:07.271555 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:07.272228 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:07.272231 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:07.272814 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:07.272822 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:07.273183 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:07.273185 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:07.273243 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:07.273725 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:07.273736 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:07.275602 (+ 1866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":104,"threads_started":1}
W20260811 15:09:07.275873 13291 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:07.270703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:55491 (local address 127.30.8.62:34091)
0811 15:09:07.270865 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:07.270868 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:07.270990 (+ 122us) server_negotiation.cc:407] Connection header received
0811 15:09:07.271102 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:07.271104 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:07.271151 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:07.271216 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:07.271642 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:07.272152 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:07.272898 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:07.273103 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:07.273170 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:07.275770 (+ 2600us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.3:55491: BlockingRecv error: recv got EOF from 127.30.8.3:55491 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 11 15:09:07 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:07 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:07.917629 13299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:07.912202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.1:35295)
0811 15:09:07.912482 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:07.912496 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:07.912607 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:07.912835 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:07.912836 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:07.913031 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:07.913137 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:07.913143 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:07.913940 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:07.913943 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:07.914498 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:07.914506 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:07.914981 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:07.914983 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:07.915048 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:07.915578 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:07.915590 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:07.917487 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":84,"threads_started":1}
W20260811 15:09:07.918097 13300 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:07.912356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:35295 (local address 127.30.8.62:34091)
0811 15:09:07.912582 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:07.912585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:07.912601 (+ 16us) server_negotiation.cc:407] Connection header received
0811 15:09:07.912683 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:07.912685 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:07.912730 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:07.912812 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:07.913243 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:07.913842 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:07.914622 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:07.914853 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:07.914970 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:07.917912 (+ 2942us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.1:35295: BlockingRecv error: recv got EOF from 127.30.8.1:35295 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":69,"threads_started":1}
Aug 11 15:09:08 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:08 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:08.106000 13301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:08.098547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.2:51915)
0811 15:09:08.098984 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:08.099004 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:08.099173 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:08.099491 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:08.099493 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:08.099698 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:08.099831 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:08.099838 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:08.100852 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:08.100857 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:08.101518 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:08.101532 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:08.102172 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:08.102176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:08.102259 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:08.103101 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:08.103117 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:08.105824 (+ 2707us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":126,"threads_started":1}
W20260811 15:09:08.106145 13300 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:08.098689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:51915 (local address 127.30.8.62:34091)
0811 15:09:08.098902 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:08.098908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:08.099124 (+ 216us) server_negotiation.cc:407] Connection header received
0811 15:09:08.099287 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:08.099290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:08.099356 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:08.099451 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:08.099943 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:08.100634 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:08.101728 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:08.101999 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:08.102106 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:08.106021 (+ 3915us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.2:51915: BlockingRecv error: recv got EOF from 127.30.8.2:51915 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 15:09:08 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:08 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:08.282020 13303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:08.276607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.3:58687)
0811 15:09:08.276908 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:08.276923 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:08.277034 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:08.277290 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:08.277293 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:08.277512 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:08.277626 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:08.277632 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:08.278388 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:08.278392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:08.278986 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:08.278995 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:08.279428 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:08.279432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:08.279507 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:08.280069 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:08.280082 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:08.281853 (+ 1771us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":222,"thread_start_us":96,"threads_started":1}
W20260811 15:09:08.282120 13300 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:08.276692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:58687 (local address 127.30.8.62:34091)
0811 15:09:08.276830 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:08.276835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:08.276988 (+ 153us) server_negotiation.cc:407] Connection header received
0811 15:09:08.277120 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:08.277124 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:08.277178 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:08.277248 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:08.277740 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:08.278284 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:08.279104 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:08.279325 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:08.279412 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:08.282003 (+ 2591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.3:58687: BlockingRecv error: recv got EOF from 127.30.8.3:58687 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 11 15:09:08 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:08 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:08.926721 13307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:08.919101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.1:39343)
0811 15:09:08.919520 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:08.919539 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:08.919702 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:08.920051 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:08.920053 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:08.920310 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:08.920451 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:08.920458 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:08.921550 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:08.921556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:08.922354 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:08.922369 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:08.922984 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:08.922990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:08.923137 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:08.924164 (+ 1027us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:08.924179 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:08.926563 (+ 2384us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":98,"threads_started":1}
W20260811 15:09:08.927052 13305 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:08.919095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:39343 (local address 127.30.8.62:34091)
0811 15:09:08.919462 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:08.919467 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:08.919689 (+ 222us) server_negotiation.cc:407] Connection header received
0811 15:09:08.919817 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:08.919821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:08.919909 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:08.920016 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:08.920605 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:08.921353 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:08.922481 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:08.922748 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:08.922882 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:08.926830 (+ 3948us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.1:39343: BlockingRecv error: recv got EOF from 127.30.8.1:39343 (error 108)
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":116,"threads_started":1}
Aug 11 15:09:09 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:09 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:09.113556 13309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:09.107333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.2:48871)
0811 15:09:09.107824 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:09.107844 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:09.107971 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:09.108333 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:09.108338 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:09.108609 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:09.108764 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:09.108773 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:09.109755 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:09.109759 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:09.110369 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:09.110382 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:09.110801 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:09.110803 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:09.110865 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:09.111431 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:09.111444 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:09.113395 (+ 1951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":374,"thread_start_us":124,"threads_started":1}
W20260811 15:09:09.113688 13305 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:09.107478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:48871 (local address 127.30.8.62:34091)
0811 15:09:09.107732 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:09.107738 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:09.107929 (+ 191us) server_negotiation.cc:407] Connection header received
0811 15:09:09.108113 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:09.108117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:09.108172 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:09.108256 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:09.108918 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:09.109598 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:09.110482 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:09.110708 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:09.110780 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:09.113562 (+ 2782us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.2:48871: BlockingRecv error: recv got EOF from 127.30.8.2:48871 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 11 15:09:09 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:09 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:09.290020 13310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:09.283221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.3:51469)
0811 15:09:09.283668 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:09.283684 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:09.283823 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:09.284194 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:09.284197 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:09.284445 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:09.284592 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:09.284599 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:09.285432 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:09.285435 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:09.286076 (+ 641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:09.286087 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:09.286887 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:09.286892 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:09.287071 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:09.287698 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:09.287712 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:09.289870 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":132,"threads_started":1}
W20260811 15:09:09.290192 13305 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:09.283302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:51469 (local address 127.30.8.62:34091)
0811 15:09:09.283502 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:09.283507 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:09.283776 (+ 269us) server_negotiation.cc:407] Connection header received
0811 15:09:09.283929 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:09.283931 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:09.283998 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:09.284100 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:09.284733 (+ 633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:09.285292 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:09.286192 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:09.286661 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:09.286782 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:09.290061 (+ 3279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.3:51469: BlockingRecv error: recv got EOF from 127.30.8.3:51469 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
Aug 11 15:09:09 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:09 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:09.932533 13311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:09.927643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.1:33011)
0811 15:09:09.927926 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:09.927938 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:09.928029 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:09.928227 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:09.928229 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:09.928417 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:09.928524 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:09.928529 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:09.929209 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:09.929211 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:09.929769 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:09.929775 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:09.930134 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:09.930136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:09.930198 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:09.930655 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:09.930666 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:09.932405 (+ 1739us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":91,"threads_started":1}
W20260811 15:09:09.932657 13312 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:09.927775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:33011 (local address 127.30.8.62:34091)
0811 15:09:09.928036 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:09.928040 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:09.928054 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:09:09.928089 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:09.928092 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:09.928133 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:09.928205 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:09.928606 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:09.929131 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:09.929845 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:09.930061 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:09.930127 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:09.932541 (+ 2414us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.1:33011: BlockingRecv error: recv got EOF from 127.30.8.1:33011 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":104,"threads_started":1}
Aug 11 15:09:10 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:10 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:10.120077 13313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:10.114599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.2:54635)
0811 15:09:10.114852 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:10.114868 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:10.115022 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:10.115273 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:10.115276 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:10.115480 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:10.115580 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:10.115586 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:10.116557 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:10.116560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:10.117131 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:10.117140 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:10.117579 (+ 439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:10.117583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:10.117669 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:10.118142 (+ 473us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:10.118153 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:10.119918 (+ 1765us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":88,"threads_started":1}
W20260811 15:09:10.120172 13312 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:10.114673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:54635 (local address 127.30.8.62:34091)
0811 15:09:10.114824 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:10.114828 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:10.114966 (+ 138us) server_negotiation.cc:407] Connection header received
0811 15:09:10.115113 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:10.115117 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:10.115176 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:10.115239 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:10.115690 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:10.116453 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:10.117248 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:10.117477 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:10.117564 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:10.120071 (+ 2507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.2:54635: BlockingRecv error: recv got EOF from 127.30.8.2:54635 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 11 15:09:10 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:10 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:10.298254 13314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:10.291231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.3:43229)
0811 15:09:10.291623 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:10.291641 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:10.291780 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:10.292137 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:10.292139 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:10.292344 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:10.292467 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:10.292473 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:10.293400 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:10.293403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:10.294152 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:10.294172 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:10.294806 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:10.294809 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:10.294895 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:10.295605 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:10.295620 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:10.298077 (+ 2457us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":124,"threads_started":1}
W20260811 15:09:10.298381 13312 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:10.291321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:43229 (local address 127.30.8.62:34091)
0811 15:09:10.291583 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:10.291589 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:10.291739 (+ 150us) server_negotiation.cc:407] Connection header received
0811 15:09:10.291906 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:10.291909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:10.291977 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:10.292065 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:10.292637 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:10.293243 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:10.294351 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:10.294626 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:10.294732 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:10.298271 (+ 3539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.3:43229: BlockingRecv error: recv got EOF from 127.30.8.3:43229 (error 108)
Metrics: {"server-negotiator.queue_time_us":114}
Aug 11 15:09:10 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:10 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:10.943625 13315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:10.933694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.1:35639)
0811 15:09:10.934165 (+ 471us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:10.934183 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:10.934340 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:10.934785 (+ 445us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:10.934788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:10.935026 (+ 238us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:10.935167 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:10.935174 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:10.936091 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:10.936096 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:10.936792 (+ 696us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:10.936805 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:10.937494 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:10.937499 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:10.937618 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:10.941342 (+ 3724us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:10.941357 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:10.943485 (+ 2128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":136,"threads_started":1}
W20260811 15:09:10.943825 13316 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:10.933881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:35639 (local address 127.30.8.62:34091)
0811 15:09:10.934193 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:10.934197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:10.934279 (+ 82us) server_negotiation.cc:407] Connection header received
0811 15:09:10.934527 (+ 248us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:10.934531 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:10.934603 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:10.934723 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:10.935309 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:10.935928 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:10.936988 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:10.937335 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:10.937452 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:10.943684 (+ 6232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.1:35639: BlockingRecv error: recv got EOF from 127.30.8.1:35639 (error 108)
Metrics: {"server-negotiator.queue_time_us":226,"thread_start_us":85,"threads_started":1}
Aug 11 15:09:11 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:11 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:11.128104 13317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:11.121351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.2:50553)
0811 15:09:11.121779 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:11.121795 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:11.121951 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:11.122459 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:11.122463 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:11.122755 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:11.122966 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:11.122974 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:11.124104 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:11.124110 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:11.124741 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:11.124755 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:11.125284 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:11.125287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:11.125356 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:11.125954 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:11.125968 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:11.127958 (+ 1990us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":127,"threads_started":1}
W20260811 15:09:11.128361 13316 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:11.121503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:50553 (local address 127.30.8.62:34091)
0811 15:09:11.121683 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:11.121688 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:11.121905 (+ 217us) server_negotiation.cc:407] Connection header received
0811 15:09:11.122139 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:11.122144 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:11.122218 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:11.122361 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:11.123112 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:11.123917 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:11.124919 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:11.125167 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:11.125267 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:11.128184 (+ 2917us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.2:50553: BlockingRecv error: recv got EOF from 127.30.8.2:50553 (error 108)
Metrics: {"server-negotiator.queue_time_us":72}
Aug 11 15:09:11 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:11 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:11.305892 13318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:11.299444 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.3:50737)
0811 15:09:11.299815 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:11.299833 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:11.299979 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:11.300284 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:11.300286 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:11.300518 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:11.300654 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:11.300660 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:11.301420 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:11.301423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:11.302045 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:11.302054 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:11.302530 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:11.302534 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:11.302614 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:11.303396 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:11.303414 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:11.305744 (+ 2330us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":117,"threads_started":1}
W20260811 15:09:11.306172 13316 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:11.299609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:50737 (local address 127.30.8.62:34091)
0811 15:09:11.299783 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:11.299787 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:11.299925 (+ 138us) server_negotiation.cc:407] Connection header received
0811 15:09:11.300074 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:11.300077 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:11.300133 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:11.300236 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:11.300758 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:11.301318 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:11.302156 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:11.302379 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:11.302455 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:11.306004 (+ 3549us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.3:50737: BlockingRecv error: recv got EOF from 127.30.8.3:50737 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 15:09:11 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:11 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:11.951198 13319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:11.944570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.1:41229)
0811 15:09:11.944864 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:11.944884 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:11.944982 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:11.945521 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:11.945525 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:11.945728 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:11.945847 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:11.945853 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:11.946792 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:11.946798 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:11.947690 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:11.947704 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:11.948244 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:11.948248 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:11.948343 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:11.948856 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:11.948868 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:11.951022 (+ 2154us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":90,"threads_started":1}
W20260811 15:09:11.951341 13321 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:11.944826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:41229 (local address 127.30.8.62:34091)
0811 15:09:11.945151 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:11.945155 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:11.945175 (+ 20us) server_negotiation.cc:407] Connection header received
0811 15:09:11.945252 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:11.945255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:11.945315 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:11.945446 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:11.946022 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:11.946605 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:11.947815 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:11.948055 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:11.948149 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:11.951179 (+ 3030us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.1:41229: BlockingRecv error: recv got EOF from 127.30.8.1:41229 (error 108)
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":103,"threads_started":1}
Aug 11 15:09:12 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:12 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:12.134286 13323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:12.129029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.2:53665)
0811 15:09:12.129326 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:12.129340 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:12.129454 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:12.129707 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:12.129711 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:12.129918 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:12.130022 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:12.130027 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:12.130761 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:12.130765 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:12.131360 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:12.131369 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:12.131800 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:12.131804 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:12.131881 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:12.132381 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:12.132392 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:12.134156 (+ 1764us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":104,"threads_started":1}
W20260811 15:09:12.134446 13321 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:12.129120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:53665 (local address 127.30.8.62:34091)
0811 15:09:12.129286 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:12.129291 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:12.129413 (+ 122us) server_negotiation.cc:407] Connection header received
0811 15:09:12.129548 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:12.129551 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:12.129603 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:12.129670 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:12.130138 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:12.130658 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:12.131480 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:12.131698 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:12.131786 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:12.134327 (+ 2541us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.2:53665: BlockingRecv error: recv got EOF from 127.30.8.2:53665 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 15:09:12 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:12 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:12.312460 13324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:12.307137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.3:43727)
0811 15:09:12.307440 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:12.307459 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:12.307555 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:12.307793 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:12.307795 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:12.307992 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:12.308098 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:12.308103 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:12.308850 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:12.308852 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:12.309400 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:12.309407 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:12.309820 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:12.309822 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:12.309880 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:12.310432 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:12.310445 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:12.312297 (+ 1852us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":109,"threads_started":1}
W20260811 15:09:12.312551 13321 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:12.307300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:43727 (local address 127.30.8.62:34091)
0811 15:09:12.307460 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:12.307464 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:12.307513 (+ 49us) server_negotiation.cc:407] Connection header received
0811 15:09:12.307644 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:12.307646 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:12.307692 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:12.307763 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:12.308231 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:12.308768 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:12.309501 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:12.309730 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:12.309810 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:12.312446 (+ 2636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.3:43727: BlockingRecv error: recv got EOF from 127.30.8.3:43727 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 15:09:12 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:12 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:12.957436 13325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:12.952182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.1:52669)
0811 15:09:12.952462 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:12.952477 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:12.952571 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:12.952838 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:12.952840 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:12.953030 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:12.953133 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:12.953138 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:12.953859 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:12.953862 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:12.954431 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:12.954439 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:12.954822 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:12.954824 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:12.954882 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:12.955430 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:12.955442 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:12.957305 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":83,"threads_started":1}
W20260811 15:09:12.957618 13326 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:12.952282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:52669 (local address 127.30.8.62:34091)
0811 15:09:12.952527 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:12.952530 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:12.952548 (+ 18us) server_negotiation.cc:407] Connection header received
0811 15:09:12.952681 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:12.952684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:12.952733 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:12.952808 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:12.953235 (+ 427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:12.953772 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:12.954516 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:12.954737 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:12.954808 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:12.957481 (+ 2673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.1:52669: BlockingRecv error: recv got EOF from 127.30.8.1:52669 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":80,"threads_started":1}
Aug 11 15:09:13 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:13 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:13.141335 13327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:13.135118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.2:59307)
0811 15:09:13.135410 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:13.135423 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:13.135527 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:13.135923 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:13.135928 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:13.136195 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:13.136360 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:13.136368 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:13.137192 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:13.137197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:13.137818 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:13.137828 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:13.138221 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:13.138223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:13.138285 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:13.138870 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:13.138883 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:13.141101 (+ 2218us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":101,"threads_started":1}
W20260811 15:09:13.141430 13326 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:13.135252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:59307 (local address 127.30.8.62:34091)
0811 15:09:13.135384 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:13.135387 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:13.135487 (+ 100us) server_negotiation.cc:407] Connection header received
0811 15:09:13.135596 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:13.135599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:13.135642 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:13.135708 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:13.136505 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:13.137038 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:13.137919 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:13.138129 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:13.138205 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:13.141307 (+ 3102us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.2:59307: BlockingRecv error: recv got EOF from 127.30.8.2:59307 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
Aug 11 15:09:13 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:13 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:13.320407 13328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:13.313418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.3:53983)
0811 15:09:13.313721 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:13.313736 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:13.313879 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:13.314182 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:13.314193 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:13.314409 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:13.314507 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:13.314512 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:13.315357 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:13.315360 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:13.315915 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:13.315923 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:13.316416 (+ 493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:13.316419 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:13.316503 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:13.317173 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:13.317187 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:13.320151 (+ 2964us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":97,"threads_started":1}
W20260811 15:09:13.320485 13326 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:13.313537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:53983 (local address 127.30.8.62:34091)
0811 15:09:13.313699 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:13.313704 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:13.313810 (+ 106us) server_negotiation.cc:407] Connection header received
0811 15:09:13.314018 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:13.314021 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:13.314067 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:13.314136 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:13.314627 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:13.315249 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:13.316032 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:13.316282 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:13.316372 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:13.320350 (+ 3978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.3:53983: BlockingRecv error: recv got EOF from 127.30.8.3:53983 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 15:09:13 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:13 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:13.965621 13330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:13.958668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.1:44705)
0811 15:09:13.959271 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:13.959289 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:13.959437 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:13.959873 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:13.959876 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:13.960087 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:13.960227 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:13.960233 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:13.961113 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:13.961116 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:13.961716 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:13.961725 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:13.962278 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:13.962281 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:13.962355 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:13.963132 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:13.963148 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:13.965410 (+ 2262us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":93,"threads_started":1}
W20260811 15:09:13.965917 13332 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:13.958832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:44705 (local address 127.30.8.62:34091)
0811 15:09:13.959237 (+ 405us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:13.959243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:13.959388 (+ 145us) server_negotiation.cc:407] Connection header received
0811 15:09:13.959612 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:13.959616 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:13.959695 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:13.959840 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:13.960364 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:13.960987 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:13.961886 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:13.962155 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:13.962248 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:13.965700 (+ 3452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.1:44705: BlockingRecv error: recv got EOF from 127.30.8.1:44705 (error 108)
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":160,"threads_started":1}
Aug 11 15:09:14 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:14 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:14.149587 13333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:14.142842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.2:37697)
0811 15:09:14.143384 (+ 542us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:14.143401 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:14.143562 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:14.143945 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:14.143949 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:14.144195 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:14.144345 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:14.144354 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:14.145168 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:14.145172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:14.145817 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:14.145826 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:14.146322 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:14.146325 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:14.146406 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:14.147074 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:14.147088 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:14.149432 (+ 2344us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":131,"threads_started":1}
W20260811 15:09:14.149931 13332 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:14.143072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:37697 (local address 127.30.8.62:34091)
0811 15:09:14.143269 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:14.143274 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:14.143515 (+ 241us) server_negotiation.cc:407] Connection header received
0811 15:09:14.143666 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:14.143669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:14.143744 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:14.143843 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:14.144479 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:14.145032 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:14.145942 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:14.146174 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:14.146272 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:14.149732 (+ 3460us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.2:37697: BlockingRecv error: recv got EOF from 127.30.8.2:37697 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 15:09:14 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:14 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:14.329151 13335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:14.321854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.3:53155)
0811 15:09:14.322227 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:14.322244 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:14.322388 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:14.322770 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:14.322773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:14.323049 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:14.323176 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:14.323182 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:14.324119 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:14.324123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:14.324747 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:14.324757 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:14.325455 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:14.325459 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:14.325540 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:14.326255 (+ 715us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:14.326271 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:14.328960 (+ 2689us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":116,"threads_started":1}
W20260811 15:09:14.329244 13332 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:14.322095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:53155 (local address 127.30.8.62:34091)
0811 15:09:14.322320 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:14.322324 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:14.322347 (+ 23us) server_negotiation.cc:407] Connection header received
0811 15:09:14.322570 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:14.322573 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:14.322643 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:14.322741 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:14.323343 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:14.323965 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:14.324950 (+ 985us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:14.325267 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:14.325430 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:14.329113 (+ 3683us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.3:53155: BlockingRecv error: recv got EOF from 127.30.8.3:53155 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Aug 11 15:09:14 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:14 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:14.972110 13336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:14.966696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.1:37227)
0811 15:09:14.967012 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:14.967026 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:14.967116 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:14.967385 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:14.967392 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:14.967591 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:14.967689 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:14.967695 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:14.968425 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:14.968428 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:14.968984 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:14.968991 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:14.969405 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:14.969407 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:14.969463 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:14.969976 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:14.969987 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:14.971962 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":87,"threads_started":1}
W20260811 15:09:14.972271 13337 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:14.966751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:37227 (local address 127.30.8.62:34091)
0811 15:09:14.967078 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:14.967081 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:14.967097 (+ 16us) server_negotiation.cc:407] Connection header received
0811 15:09:14.967203 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:14.967206 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:14.967268 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:14.967359 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:14.967778 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:14.968340 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:14.969084 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:14.969313 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:14.969392 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:14.972134 (+ 2742us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.1:37227: BlockingRecv error: recv got EOF from 127.30.8.1:37227 (error 108)
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":85,"threads_started":1}
Aug 11 15:09:15 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:15 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:15.155958 13338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:15.150523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.2:58961)
0811 15:09:15.150793 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:15.150806 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:15.150959 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:15.151212 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:15.151214 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:15.151397 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:15.151535 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:15.151541 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:15.152295 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:15.152298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:15.152957 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:15.152968 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:15.153405 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:15.153408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:15.153476 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:15.154014 (+ 538us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:15.154026 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:15.155838 (+ 1812us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":75,"threads_started":1}
W20260811 15:09:15.156109 13337 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:15.150688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:58961 (local address 127.30.8.62:34091)
0811 15:09:15.150844 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:15.150848 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:15.150862 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:09:15.151079 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:15.151081 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:15.151128 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:15.151207 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:15.151633 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:15.152208 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:15.153069 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:15.153297 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:15.153386 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:15.155979 (+ 2593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.2:58961: BlockingRecv error: recv got EOF from 127.30.8.2:58961 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
Aug 11 15:09:15 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:15 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:15.335832 13339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:15.330033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.3:49433)
0811 15:09:15.330944 (+ 911us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:15.330959 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:15.331074 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:15.331354 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:15.331357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:15.331542 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:15.331647 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:15.331653 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:15.332353 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:15.332356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:15.332919 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:15.332927 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:15.333288 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:15.333290 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:15.333349 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:15.333882 (+ 533us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:15.333897 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:15.335708 (+ 1811us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":799,"thread_start_us":109,"threads_started":1}
W20260811 15:09:15.335942 13337 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:15.330346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:49433 (local address 127.30.8.62:34091)
0811 15:09:15.330472 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:15.330475 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:15.331043 (+ 568us) server_negotiation.cc:407] Connection header received
0811 15:09:15.331165 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:15.331168 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:15.331232 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:15.331307 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:15.331735 (+ 428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:15.332263 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:15.333004 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:15.333210 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:15.333274 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:15.335838 (+ 2564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.3:49433: BlockingRecv error: recv got EOF from 127.30.8.3:49433 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 11 15:09:15 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:15 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:15.983481 13342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:15.973285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.1:57491)
0811 15:09:15.973680 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:15.973696 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:15.973840 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:15.974170 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:15.974173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:15.974414 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:15.974553 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:15.974559 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:15.975682 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:15.975686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:15.976447 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:15.976461 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:15.977109 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:15.977113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:15.977201 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:15.980575 (+ 3374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:15.980592 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:15.983342 (+ 2750us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":128,"threads_started":1}
W20260811 15:09:15.983741 13343 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:15.973420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:57491 (local address 127.30.8.62:34091)
0811 15:09:15.973799 (+ 379us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:15.973803 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:15.973823 (+ 20us) server_negotiation.cc:407] Connection header received
0811 15:09:15.973947 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:15.973950 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:15.974000 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:15.974120 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:15.974767 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:15.975481 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:15.976605 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:15.976951 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:15.977075 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:15.983592 (+ 6517us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.1:57491: BlockingRecv error: recv got EOF from 127.30.8.1:57491 (error 108)
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":92,"threads_started":1}
Aug 11 15:09:16 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:16 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:16.162744 13346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:16.156837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.2:48137)
0811 15:09:16.157136 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:16.157154 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:16.157295 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:16.157560 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:16.157563 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:16.157770 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:16.157872 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:16.157878 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:16.158641 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:16.158645 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:16.159248 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:16.159257 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:16.159694 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:16.159697 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:16.159788 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:16.160313 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:16.160326 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:16.162559 (+ 2233us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":91,"threads_started":1}
W20260811 15:09:16.162837 13343 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:16.157062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:48137 (local address 127.30.8.62:34091)
0811 15:09:16.157256 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:16.157261 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:16.157280 (+ 19us) server_negotiation.cc:407] Connection header received
0811 15:09:16.157398 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:16.157402 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:16.157452 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:16.157522 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:16.157994 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:16.158534 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:16.159366 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:16.159584 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:16.159675 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:16.162725 (+ 3050us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.2:48137: BlockingRecv error: recv got EOF from 127.30.8.2:48137 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 11 15:09:16 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:16 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.3@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:16.342186 13350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:16.336684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.3:35111)
0811 15:09:16.336988 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:16.337001 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:16.337128 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:16.337375 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:16.337378 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:16.337599 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:16.337699 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:16.337705 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:16.338453 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:16.338456 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:16.339060 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:16.339070 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:16.339503 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:16.339507 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:16.339584 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:16.340130 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:16.340142 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:16.341979 (+ 1837us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":119,"threads_started":1}
W20260811 15:09:16.342247 13343 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:16.336771 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:35111 (local address 127.30.8.62:34091)
0811 15:09:16.336907 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:16.336912 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:16.337069 (+ 157us) server_negotiation.cc:407] Connection header received
0811 15:09:16.337218 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:16.337221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:16.337271 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:16.337336 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:16.337812 (+ 476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:16.338344 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:16.339177 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:16.339400 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:16.339486 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:16.342138 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.3:35111: BlockingRecv error: recv got EOF from 127.30.8.3:35111 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 11 15:09:16 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:16 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.1@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:16.990854 13351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:16.984621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.1:58659)
0811 15:09:16.984944 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:16.984963 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:16.985083 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:16.985571 (+ 488us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:16.985574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:16.985810 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:16.985958 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:16.985967 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:16.987127 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:16.987130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:16.987728 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:16.987739 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:16.988267 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:16.988270 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:16.988327 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:16.988867 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:16.988878 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:16.990727 (+ 1849us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":104,"threads_started":1}
W20260811 15:09:16.991075 13352 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:16.984870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:58659 (local address 127.30.8.62:34091)
0811 15:09:16.985266 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:16.985270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:16.985289 (+ 19us) server_negotiation.cc:407] Connection header received
0811 15:09:16.985348 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:16.985351 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:16.985407 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:16.985512 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:16.986232 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:16.986989 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:16.987879 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:16.988151 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:16.988255 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:16.990897 (+ 2642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.1:58659: BlockingRecv error: recv got EOF from 127.30.8.1:58659 (error 108)
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":150,"threads_started":1}
Aug 11 15:09:17 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
Aug 11 15:09:17 dist-test-slave-1vmg krb5kdc[11702](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.8.2@KRBTEST.COM for oryx/127.30.8.62@KRBTEST.COM, Server not found in Kerberos database
I20260811 15:09:17.170431 13354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:17.163850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:34091 (local address 127.30.8.2:37121)
0811 15:09:17.164325 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:17.164347 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:17.164521 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:17.164829 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:17.164832 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:17.165081 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:17.165217 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:17.165226 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:17.166222 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:17.166226 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:17.167066 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:17.167077 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:17.167574 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:17.167577 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:17.167657 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:17.168291 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:17.168306 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:17.170246 (+ 1940us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.8.62:34091: Server oryx/127.30.8.62@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":98,"threads_started":1}
W20260811 15:09:17.170512 13352 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:17.164111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:37121 (local address 127.30.8.62:34091)
0811 15:09:17.164331 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:17.164337 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:17.164450 (+ 113us) server_negotiation.cc:407] Connection header received
0811 15:09:17.164637 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:17.164640 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:17.164695 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:17.164775 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:17.165377 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:17.166081 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:17.167199 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:17.167435 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:17.167526 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:17.170406 (+ 2880us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.8.2:37121: BlockingRecv error: recv got EOF from 127.30.8.2:37121 (error 108)
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 15:09:17.234894 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 12619
I20260811 15:09:17.253312 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 12861
I20260811 15:09:17.270092 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 13054
I20260811 15:09:17.281198 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 12505
2026-08-11T15:09:17Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18234 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T15:09:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:09:17Z Disabled control of system clock
I20260811 15:09:17.320703 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:35543
--webserver_interface=127.30.8.62
--webserver_port=0
--builtin_ntp_servers=127.30.8.20:36257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:35543
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 15:09:17.441609 13358 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:09:17.441929 13358 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:09:17.441965 13358 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:09:17.446117 13358 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:09:17.446200 13358 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:09:17.446219 13358 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:09:17.446237 13358 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:09:17.446269 13358 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:09:17.451584 13358 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:35543
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:35543
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13358
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:09:17.452862 13358 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:09:17.454286 13358 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:09:17.463304 13371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:17.463368 13368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:17.463372 13358 server_base.cc:1034] running on GCE node
W20260811 15:09:17.463312 13367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:17.464185 13358 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:09:17.465394 13358 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:09:17.466562 13358 hybrid_clock.cc:648] HybridClock initialized: now 1786460957466545 us; error 36 us; skew 500 ppm
I20260811 15:09:17.468609 13358 webserver.cc:460] Webserver started at http://127.30.8.62:45927/ using document root <none> and password file <none>
I20260811 15:09:17.469242 13358 fs_manager.cc:359] Metadata directory not provided
I20260811 15:09:17.469317 13358 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:09:17.469532 13358 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:09:17.471881 13358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "e6b748edcfa54cf8867f1e05f549481d"
format_stamp: "Formatted at 2026-08-11 15:09:17 on dist-test-slave-1vmg"
server_key: "78f8af053e49cd8f56071d8344794c80"
server_key_iv: "3a7b0ca3ec6f7ad8c2436d61ae547925"
server_key_version: "encryptionkey@0"
I20260811 15:09:17.472466 13358 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "e6b748edcfa54cf8867f1e05f549481d"
format_stamp: "Formatted at 2026-08-11 15:09:17 on dist-test-slave-1vmg"
server_key: "78f8af053e49cd8f56071d8344794c80"
server_key_iv: "3a7b0ca3ec6f7ad8c2436d61ae547925"
server_key_version: "encryptionkey@0"
I20260811 15:09:17.476338 13358 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:09:17.483273 13381 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:17.485157 13358 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.004s sys 0.000s
I20260811 15:09:17.485337 13358 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "e6b748edcfa54cf8867f1e05f549481d"
format_stamp: "Formatted at 2026-08-11 15:09:17 on dist-test-slave-1vmg"
server_key: "78f8af053e49cd8f56071d8344794c80"
server_key_iv: "3a7b0ca3ec6f7ad8c2436d61ae547925"
server_key_version: "encryptionkey@0"
I20260811 15:09:17.485508 13358 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:09:17.496788 13358 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:09:17.497498 13358 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:09:17.497663 13358 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:09:17.504717 13358 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:35543
I20260811 15:09:17.504710 13459 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:35543 every 8 connection(s)
I20260811 15:09:17.505797 13358 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:09:17.507191 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 13358
I20260811 15:09:17.507287 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:09:17.507555 30752 external_mini_cluster.cc:1428] Setting key 52d2852f1463e7a57c2d37a96e5366aa
I20260811 15:09:17.509513 13461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:09:17.516896 13461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d: Bootstrap starting.
I20260811 15:09:17.518203 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:17.508922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32858 (local address 127.30.8.62:35543)
0811 15:09:17.509641 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:17.509662 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:17.509706 (+ 44us) server_negotiation.cc:407] Connection header received
0811 15:09:17.510674 (+ 968us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:17.510699 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:17.511236 (+ 537us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:17.511624 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:17.512080 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:17.513582 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:17.514454 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:17.515172 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:17.515334 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:17.516125 (+ 791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:09:17.516133 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:09:17.516149 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:09:17.516670 (+ 521us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:09:17.516747 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:09:17.516775 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:09:17.516879 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:17.516959 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 15:09:17.517241 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":127,"threads_started":1}
I20260811 15:09:17.519642 13461 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d: Neither blocks nor log segments found. Creating new log.
I20260811 15:09:17.520589 13461 log.cc:824] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d: Log is configured to *not* fsync() on all Append() calls
I20260811 15:09:17.522571 13461 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d: No bootstrap required, opened a new log
I20260811 15:09:17.526304 13461 raft_consensus.cc:348] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b748edcfa54cf8867f1e05f549481d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 35543 } }
I20260811 15:09:17.526552 13461 raft_consensus.cc:374] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:09:17.526614 13461 raft_consensus.cc:729] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e6b748edcfa54cf8867f1e05f549481d, State: Initialized, Role: FOLLOWER
I20260811 15:09:17.527086 13461 consensus_queue.cc:260] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b748edcfa54cf8867f1e05f549481d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 35543 } }
I20260811 15:09:17.527230 13461 raft_consensus.cc:388] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:09:17.527297 13461 raft_consensus.cc:482] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:09:17.527400 13461 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:09:17.528398 13461 raft_consensus.cc:504] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b748edcfa54cf8867f1e05f549481d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 35543 } }
I20260811 15:09:17.528744 13461 leader_election.cc:302] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e6b748edcfa54cf8867f1e05f549481d; no voters:
I20260811 15:09:17.529103 13461 leader_election.cc:288] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:09:17.529233 13470 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:09:17.529470 13470 raft_consensus.cc:686] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d [term 1 LEADER]: Becoming Leader. State: Replica: e6b748edcfa54cf8867f1e05f549481d, State: Running, Role: LEADER
I20260811 15:09:17.529874 13470 consensus_queue.cc:237] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b748edcfa54cf8867f1e05f549481d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 35543 } }
I20260811 15:09:17.530438 13461 sys_catalog.cc:564] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:09:17.531697 13471 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e6b748edcfa54cf8867f1e05f549481d. Latest consensus state: current_term: 1 leader_uuid: "e6b748edcfa54cf8867f1e05f549481d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b748edcfa54cf8867f1e05f549481d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 35543 } } }
I20260811 15:09:17.531836 13471 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d [sys.catalog]: This master's current role is: LEADER
I20260811 15:09:17.532156 13474 sys_catalog.cc:455] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e6b748edcfa54cf8867f1e05f549481d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e6b748edcfa54cf8867f1e05f549481d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 35543 } } }
I20260811 15:09:17.532321 13474 sys_catalog.cc:458] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d [sys.catalog]: This master's current role is: LEADER
I20260811 15:09:17.532243 13489 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:09:17.535854 13489 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:09:17.541352 13489 catalog_manager.cc:1347] Generated new cluster ID: d0423a0dfcad4b65a601edac025a61d3
I20260811 15:09:17.541437 13489 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:09:17.555953 13489 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:09:17.556978 13489 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:09:17.561111 13489 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e6b748edcfa54cf8867f1e05f549481d: Generated new TSK 0
I20260811 15:09:17.561841 13489 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:09:17.566560 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:35543
--builtin_ntp_servers=127.30.8.20:36257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 15:09:17.685549 13502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:09:17.685837 13502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:09:17.685876 13502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:09:17.689946 13502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:09:17.690018 13502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:09:17.690079 13502 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:09:17.695108 13502 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:35543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:09:17.696228 13502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:09:17.697628 13502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:09:17.707470 13510 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:17.707471 13512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:17.707508 13502 server_base.cc:1034] running on GCE node
W20260811 15:09:17.707470 13509 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:17.708163 13502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:09:17.708819 13502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:09:17.709985 13502 hybrid_clock.cc:648] HybridClock initialized: now 1786460957709963 us; error 35 us; skew 500 ppm
I20260811 15:09:17.712008 13502 webserver.cc:460] Webserver started at http://127.30.8.1:46569/ using document root <none> and password file <none>
I20260811 15:09:17.712607 13502 fs_manager.cc:359] Metadata directory not provided
I20260811 15:09:17.712683 13502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:09:17.712913 13502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:09:17.715299 13502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "9b9b6e4ec2644c23869d4fdae2ef933b"
format_stamp: "Formatted at 2026-08-11 15:09:17 on dist-test-slave-1vmg"
server_key: "a9e9bd3a8e07a7e5c812be3c36cd4d93"
server_key_iv: "c49e4f1738bc28a7bd44a8cf8a35cb29"
server_key_version: "encryptionkey@0"
I20260811 15:09:17.715869 13502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "9b9b6e4ec2644c23869d4fdae2ef933b"
format_stamp: "Formatted at 2026-08-11 15:09:17 on dist-test-slave-1vmg"
server_key: "a9e9bd3a8e07a7e5c812be3c36cd4d93"
server_key_iv: "c49e4f1738bc28a7bd44a8cf8a35cb29"
server_key_version: "encryptionkey@0"
I20260811 15:09:17.719764 13502 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:09:17.722437 13520 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:17.723711 13502 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:09:17.723847 13502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "9b9b6e4ec2644c23869d4fdae2ef933b"
format_stamp: "Formatted at 2026-08-11 15:09:17 on dist-test-slave-1vmg"
server_key: "a9e9bd3a8e07a7e5c812be3c36cd4d93"
server_key_iv: "c49e4f1738bc28a7bd44a8cf8a35cb29"
server_key_version: "encryptionkey@0"
I20260811 15:09:17.723973 13502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:09:17.745848 13502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:09:17.746620 13502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:09:17.746822 13502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:09:17.747491 13502 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:09:17.748554 13502 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:09:17.748602 13502 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:17.748649 13502 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:09:17.748687 13502 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:17.757835 13502 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:39733
I20260811 15:09:17.757859 13695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:39733 every 8 connection(s)
I20260811 15:09:17.758812 13502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:09:17.764668 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 13502
I20260811 15:09:17.764783 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:09:17.765041 30752 external_mini_cluster.cc:1428] Setting key 83c39710a42d8dcfe23894161ce767b9
I20260811 15:09:17.767204 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:35543
--builtin_ntp_servers=127.30.8.20:36257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 15:09:17.768747 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:17.760741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:39805 (local address 127.30.8.62:35543)
0811 15:09:17.760880 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:17.760884 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:17.761977 (+ 1093us) server_negotiation.cc:407] Connection header received
0811 15:09:17.763418 (+ 1441us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:17.763422 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:17.763488 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:17.763562 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:17.764545 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:17.765170 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:17.766581 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:17.766810 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:17.766884 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:17.768268 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:09:17.768273 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:09:17.768275 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:09:17.768291 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:09:17.768316 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:09:17.768326 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:09:17.768391 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:17.768548 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 15:09:17.768618 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 15:09:17.769825 13699 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:17.761136 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:35543 (local address 127.30.8.1:39805)
0811 15:09:17.761822 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:17.761863 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:17.763196 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:17.763749 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:17.763756 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:17.763840 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:17.764364 (+ 524us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:17.764383 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:17.765305 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:17.765309 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:17.766451 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:17.766461 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:17.766999 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:17.767004 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:17.767225 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:17.767935 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:09:17.767937 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:17.768000 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:09:17.768091 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:09:17.768406 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:09:17.768415 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:09:17.768421 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 15:09:17.768549 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 15:09:17.768973 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":131,"threads_started":1}
I20260811 15:09:17.771123 13696 heartbeater.cc:347] Connected to a master server at 127.30.8.62:35543
I20260811 15:09:17.771402 13696 heartbeater.cc:464] Registering TS with master...
I20260811 15:09:17.772113 13696 heartbeater.cc:511] Master 127.30.8.62:35543 requested a full tablet report, sending...
I20260811 15:09:17.774025 13406 ts_manager.cc:194] Registered new tserver with Master: 9b9b6e4ec2644c23869d4fdae2ef933b (127.30.8.1:39733)
I20260811 15:09:17.775782 13406 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.8.1:39805
W20260811 15:09:17.886277 13700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:09:17.886583 13700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:09:17.886618 13700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:09:17.890704 13700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:09:17.890789 13700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:09:17.890846 13700 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:09:17.895758 13700 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:35543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:09:17.896910 13700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:09:17.898307 13700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:09:17.908236 13706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:17.908306 13700 server_base.cc:1034] running on GCE node
W20260811 15:09:17.908257 13712 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:17.908242 13707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:17.908861 13700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:09:17.909555 13700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:09:17.910727 13700 hybrid_clock.cc:648] HybridClock initialized: now 1786460957910702 us; error 30 us; skew 500 ppm
I20260811 15:09:17.912662 13700 webserver.cc:460] Webserver started at http://127.30.8.2:39825/ using document root <none> and password file <none>
I20260811 15:09:17.913270 13700 fs_manager.cc:359] Metadata directory not provided
I20260811 15:09:17.913342 13700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:09:17.913569 13700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:09:17.915783 13700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "135debce2c69436db0e0dd69366947c7"
format_stamp: "Formatted at 2026-08-11 15:09:17 on dist-test-slave-1vmg"
server_key: "e0e907ae6cb1976ce731ff86d46bb13b"
server_key_iv: "5b076d8fb4ed2ceb9b87394229d57a91"
server_key_version: "encryptionkey@0"
I20260811 15:09:17.916339 13700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "135debce2c69436db0e0dd69366947c7"
format_stamp: "Formatted at 2026-08-11 15:09:17 on dist-test-slave-1vmg"
server_key: "e0e907ae6cb1976ce731ff86d46bb13b"
server_key_iv: "5b076d8fb4ed2ceb9b87394229d57a91"
server_key_version: "encryptionkey@0"
I20260811 15:09:17.920179 13700 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:09:17.922794 13725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:17.924049 13700 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:09:17.924189 13700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "135debce2c69436db0e0dd69366947c7"
format_stamp: "Formatted at 2026-08-11 15:09:17 on dist-test-slave-1vmg"
server_key: "e0e907ae6cb1976ce731ff86d46bb13b"
server_key_iv: "5b076d8fb4ed2ceb9b87394229d57a91"
server_key_version: "encryptionkey@0"
I20260811 15:09:17.924301 13700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:09:17.941998 13700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:09:17.942785 13700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:09:17.943003 13700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:09:17.943632 13700 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:09:17.944643 13700 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:09:17.944693 13700 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:17.944749 13700 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:09:17.944779 13700 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:17.953365 13700 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:37615
I20260811 15:09:17.953389 13895 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:37615 every 8 connection(s)
I20260811 15:09:17.954337 13700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:09:17.955194 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 13700
I20260811 15:09:17.955317 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:09:17.955623 30752 external_mini_cluster.cc:1428] Setting key cac32d84469bbd46cd1bd5acfe419b11
I20260811 15:09:17.957762 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:35543
--builtin_ntp_servers=127.30.8.20:36257
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 15:09:17.965273 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:17.956484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:55173 (local address 127.30.8.62:35543)
0811 15:09:17.956631 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:17.956636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:17.957980 (+ 1344us) server_negotiation.cc:407] Connection header received
0811 15:09:17.959854 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:17.959858 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:17.959919 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:17.959997 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:17.960928 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:17.961523 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:17.963218 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:17.963433 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:17.963511 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:17.964817 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:09:17.964820 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:09:17.964823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:09:17.964836 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:09:17.964855 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:09:17.964862 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:09:17.964917 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:17.965089 (+ 172us) server_negotiation.cc:299] Negotiation successful
0811 15:09:17.965152 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 15:09:17.966696 13899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:17.957015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:35543 (local address 127.30.8.2:55173)
0811 15:09:17.957822 (+ 807us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:17.957881 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:17.959553 (+ 1672us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:17.960162 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:17.960169 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:17.960258 (+ 89us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:17.960786 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:17.960801 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:17.961675 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:17.961679 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:17.963101 (+ 1422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:17.963115 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:17.963605 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:17.963611 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:17.963829 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:17.964481 (+ 652us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:09:17.964484 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:17.964546 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:09:17.964655 (+ 109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:09:17.964941 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:09:17.964950 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:09:17.964957 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 15:09:17.965100 (+ 143us) client_negotiation.cc:240] Negotiation successful
0811 15:09:17.965561 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":110,"threads_started":1}
I20260811 15:09:17.968003 13896 heartbeater.cc:347] Connected to a master server at 127.30.8.62:35543
I20260811 15:09:17.968298 13896 heartbeater.cc:464] Registering TS with master...
I20260811 15:09:17.969038 13896 heartbeater.cc:511] Master 127.30.8.62:35543 requested a full tablet report, sending...
I20260811 15:09:17.970284 13406 ts_manager.cc:194] Registered new tserver with Master: 135debce2c69436db0e0dd69366947c7 (127.30.8.2:37615)
I20260811 15:09:17.970943 13406 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.8.2:55173
W20260811 15:09:18.084473 13900 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:09:18.084789 13900 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:09:18.084827 13900 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:09:18.089188 13900 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:09:18.089278 13900 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:09:18.089354 13900 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:09:18.094591 13900 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36257
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:35543
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13900
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:09:18.095939 13900 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:09:18.097440 13900 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 15:09:18.109055 13900 server_base.cc:1034] running on GCE node
W20260811 15:09:18.108953 13906 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:18.108939 13915 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:18.108939 13908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:18.110038 13900 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:09:18.111123 13900 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:09:18.112300 13900 hybrid_clock.cc:648] HybridClock initialized: now 1786460958112283 us; error 43 us; skew 500 ppm
I20260811 15:09:18.114393 13900 webserver.cc:460] Webserver started at http://127.30.8.3:38925/ using document root <none> and password file <none>
I20260811 15:09:18.115077 13900 fs_manager.cc:359] Metadata directory not provided
I20260811 15:09:18.115164 13900 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:09:18.115401 13900 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:09:18.117749 13900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "a9b306cdd9c6482cbb13c9e2ce4d885e"
format_stamp: "Formatted at 2026-08-11 15:09:18 on dist-test-slave-1vmg"
server_key: "29b0352a95416adfe961893e4250ae05"
server_key_iv: "4adb53f9e587a130189886cb1de70a4b"
server_key_version: "encryptionkey@0"
I20260811 15:09:18.118366 13900 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "a9b306cdd9c6482cbb13c9e2ce4d885e"
format_stamp: "Formatted at 2026-08-11 15:09:18 on dist-test-slave-1vmg"
server_key: "29b0352a95416adfe961893e4250ae05"
server_key_iv: "4adb53f9e587a130189886cb1de70a4b"
server_key_version: "encryptionkey@0"
I20260811 15:09:18.122597 13900 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 15:09:18.126070 13933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:18.127938 13900 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.000s sys 0.002s
I20260811 15:09:18.128084 13900 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "a9b306cdd9c6482cbb13c9e2ce4d885e"
format_stamp: "Formatted at 2026-08-11 15:09:18 on dist-test-slave-1vmg"
server_key: "29b0352a95416adfe961893e4250ae05"
server_key_iv: "4adb53f9e587a130189886cb1de70a4b"
server_key_version: "encryptionkey@0"
I20260811 15:09:18.128223 13900 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:09:18.146418 13900 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:09:18.147349 13900 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:09:18.147541 13900 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:09:18.148207 13900 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:09:18.149293 13900 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:09:18.149344 13900 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:18.149386 13900 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:09:18.149400 13900 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:18.158768 13900 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:42293
I20260811 15:09:18.158794 14107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:42293 every 8 connection(s)
I20260811 15:09:18.159829 13900 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:09:18.167078 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 13900
I20260811 15:09:18.167182 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:09:18.167429 30752 external_mini_cluster.cc:1428] Setting key 039a1f00bf6b40f5c34ba314687a842f
I20260811 15:09:18.169726 13467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:18.161875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:49787 (local address 127.30.8.62:35543)
0811 15:09:18.162019 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:18.162023 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:18.163221 (+ 1198us) server_negotiation.cc:407] Connection header received
0811 15:09:18.164648 (+ 1427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:18.164651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:18.164704 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:18.164787 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:18.165770 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:18.166326 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:18.167618 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:18.167888 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:18.167991 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:18.169221 (+ 1230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:09:18.169224 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:09:18.169226 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:09:18.169239 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:09:18.169259 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:09:18.169267 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:09:18.169330 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:18.169507 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 15:09:18.169571 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 15:09:18.170717 14110 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:18.162301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:35543 (local address 127.30.8.3:49787)
0811 15:09:18.163061 (+ 760us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:18.163105 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:18.164425 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:18.164954 (+ 529us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:18.164962 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:18.165052 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:18.165585 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:18.165598 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:18.166428 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:18.166430 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:18.167470 (+ 1040us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:18.167484 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:18.168058 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:18.168062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:18.168232 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:18.168927 (+ 695us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:09:18.168929 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:18.168976 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:09:18.169052 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:09:18.169343 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:09:18.169353 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:09:18.169359 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 15:09:18.169497 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 15:09:18.169788 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":116,"threads_started":1}
I20260811 15:09:18.172129 14108 heartbeater.cc:347] Connected to a master server at 127.30.8.62:35543
I20260811 15:09:18.172418 14108 heartbeater.cc:464] Registering TS with master...
I20260811 15:09:18.173161 14108 heartbeater.cc:511] Master 127.30.8.62:35543 requested a full tablet report, sending...
I20260811 15:09:18.180838 13406 ts_manager.cc:194] Registered new tserver with Master: a9b306cdd9c6482cbb13c9e2ce4d885e (127.30.8.3:42293)
I20260811 15:09:18.181595 13406 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.8.3:49787
I20260811 15:09:18.187391 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 15:09:18.190624 13467 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:18.189741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32860 (local address 127.30.8.62:35543)
0811 15:09:18.189880 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:18.189883 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:18.189917 (+ 34us) server_negotiation.cc:407] Connection header received
0811 15:09:18.190071 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:18.190073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:18.190130 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:18.190193 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:18.190478 (+ 285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:32860: BlockingRecv error: recv got EOF from 127.0.0.1:32860 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 15:09:18.190898 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 13502
I20260811 15:09:18.228188 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 13700
I20260811 15:09:18.251065 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 13900
I20260811 15:09:18.273623 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 13358
2026-08-11T15:09:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1005 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T15:09:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:09:18Z Disabled control of system clock
I20260811 15:09:18.330505 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:39573
--webserver_interface=127.30.8.62
--webserver_port=0
--builtin_ntp_servers=127.30.8.20:45303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:39573
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 15:09:18.454944 14127 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:09:18.455281 14127 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:09:18.455319 14127 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:09:18.459616 14127 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:09:18.459697 14127 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:09:18.459713 14127 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:09:18.459749 14127 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:09:18.459769 14127 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:09:18.464951 14127 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:45303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:39573
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:39573
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14127
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:09:18.466235 14127 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:09:18.467813 14127 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:09:18.477306 14137 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:18.477406 14140 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:18.477826 14144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:18.477946 14127 server_base.cc:1034] running on GCE node
I20260811 15:09:18.479110 14127 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:09:18.480865 14127 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:09:18.482134 14127 hybrid_clock.cc:648] HybridClock initialized: now 1786460958482084 us; error 74 us; skew 500 ppm
I20260811 15:09:18.484858 14127 webserver.cc:460] Webserver started at http://127.30.8.62:43341/ using document root <none> and password file <none>
I20260811 15:09:18.485564 14127 fs_manager.cc:359] Metadata directory not provided
I20260811 15:09:18.485646 14127 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:09:18.485877 14127 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:09:18.488307 14127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "a4ac7e70947a414f99db4ab2e35aecf7"
format_stamp: "Formatted at 2026-08-11 15:09:18 on dist-test-slave-1vmg"
server_key: "354d4565522a0b0d43baf1fa34d4c312"
server_key_iv: "67f373ab4f79b818b0d78d7bbc99c0e0"
server_key_version: "encryptionkey@0"
I20260811 15:09:18.488929 14127 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "a4ac7e70947a414f99db4ab2e35aecf7"
format_stamp: "Formatted at 2026-08-11 15:09:18 on dist-test-slave-1vmg"
server_key: "354d4565522a0b0d43baf1fa34d4c312"
server_key_iv: "67f373ab4f79b818b0d78d7bbc99c0e0"
server_key_version: "encryptionkey@0"
I20260811 15:09:18.493587 14127 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:09:18.496803 14152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:18.498737 14127 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260811 15:09:18.498903 14127 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "a4ac7e70947a414f99db4ab2e35aecf7"
format_stamp: "Formatted at 2026-08-11 15:09:18 on dist-test-slave-1vmg"
server_key: "354d4565522a0b0d43baf1fa34d4c312"
server_key_iv: "67f373ab4f79b818b0d78d7bbc99c0e0"
server_key_version: "encryptionkey@0"
I20260811 15:09:18.499086 14127 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:09:18.504114 14127 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:09:18.504807 14127 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:09:18.504983 14127 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:09:18.513496 14235 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:39573 every 8 connection(s)
I20260811 15:09:18.513494 14127 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:39573
I20260811 15:09:18.514635 14127 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:09:18.518383 14236 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:09:18.519506 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 14127
I20260811 15:09:18.519629 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:09:18.519968 30752 external_mini_cluster.cc:1428] Setting key 1f676f4f780021276990dbd01efee938
I20260811 15:09:18.526952 14236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7: Bootstrap starting.
I20260811 15:09:18.528085 14240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:18.522337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59588 (local address 127.30.8.62:39573)
0811 15:09:18.523394 (+ 1057us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:18.523418 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:18.523473 (+ 55us) server_negotiation.cc:407] Connection header received
0811 15:09:18.524383 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:18.524411 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:18.524935 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:18.525377 (+ 442us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:18.526394 (+ 1017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:09:18.526400 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:09:18.526416 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:09:18.526429 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:09:18.526508 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:09:18.526533 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:09:18.526720 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:18.526800 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 15:09:18.527131 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":468,"thread_start_us":208,"threads_started":1}
I20260811 15:09:18.529716 14236 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7: Neither blocks nor log segments found. Creating new log.
I20260811 15:09:18.530646 14236 log.cc:824] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7: Log is configured to *not* fsync() on all Append() calls
I20260811 15:09:18.533195 14236 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7: No bootstrap required, opened a new log
I20260811 15:09:18.537503 14236 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4ac7e70947a414f99db4ab2e35aecf7" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 39573 } }
I20260811 15:09:18.537803 14236 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:09:18.537974 14236 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4ac7e70947a414f99db4ab2e35aecf7, State: Initialized, Role: FOLLOWER
I20260811 15:09:18.538460 14236 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4ac7e70947a414f99db4ab2e35aecf7" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 39573 } }
I20260811 15:09:18.538626 14236 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:09:18.538702 14236 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:09:18.538817 14236 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:09:18.539911 14236 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4ac7e70947a414f99db4ab2e35aecf7" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 39573 } }
I20260811 15:09:18.540288 14236 leader_election.cc:302] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4ac7e70947a414f99db4ab2e35aecf7; no voters:
I20260811 15:09:18.540643 14236 leader_election.cc:288] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:09:18.540913 14242 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:09:18.541287 14242 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7 [term 1 LEADER]: Becoming Leader. State: Replica: a4ac7e70947a414f99db4ab2e35aecf7, State: Running, Role: LEADER
I20260811 15:09:18.541750 14242 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4ac7e70947a414f99db4ab2e35aecf7" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 39573 } }
I20260811 15:09:18.542143 14236 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:09:18.544178 14244 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4ac7e70947a414f99db4ab2e35aecf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4ac7e70947a414f99db4ab2e35aecf7" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 39573 } } }
I20260811 15:09:18.544368 14244 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7 [sys.catalog]: This master's current role is: LEADER
I20260811 15:09:18.544768 14245 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4ac7e70947a414f99db4ab2e35aecf7. Latest consensus state: current_term: 1 leader_uuid: "a4ac7e70947a414f99db4ab2e35aecf7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4ac7e70947a414f99db4ab2e35aecf7" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 39573 } } }
I20260811 15:09:18.545126 14245 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7 [sys.catalog]: This master's current role is: LEADER
W20260811 15:09:18.547637 14265 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:09:18.547727 14265 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:09:18.547902 14266 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:09:18.549078 14266 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:09:18.556511 14266 catalog_manager.cc:1347] Generated new cluster ID: 488e8333b90b417d9c36e0f57beef811
I20260811 15:09:18.556624 14266 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:09:18.581097 14266 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:09:18.581513 14266 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:09:18.595253 14266 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4ac7e70947a414f99db4ab2e35aecf7: Generated new TSK 0
I20260811 15:09:18.596377 14266 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:09:18.600549 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:39573
--builtin_ntp_servers=127.30.8.20:45303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 15:09:18.743582 14272 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:09:18.744009 14272 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:09:18.744078 14272 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:09:18.748622 14272 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:09:18.748703 14272 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:09:18.748769 14272 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:09:18.754953 14272 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:45303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:39573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14272
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:09:18.756364 14272 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:09:18.757954 14272 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:09:18.770308 14283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:18.770560 14272 server_base.cc:1034] running on GCE node
W20260811 15:09:18.770390 14282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:18.770280 14286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:18.771392 14272 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:09:18.772224 14272 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:09:18.773404 14272 hybrid_clock.cc:648] HybridClock initialized: now 1786460958773385 us; error 35 us; skew 500 ppm
I20260811 15:09:18.775702 14272 webserver.cc:460] Webserver started at http://127.30.8.1:35419/ using document root <none> and password file <none>
I20260811 15:09:18.776357 14272 fs_manager.cc:359] Metadata directory not provided
I20260811 15:09:18.776435 14272 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:09:18.776679 14272 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:09:18.779348 14272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "9bd694fde25346b5b538559f64d6717e"
format_stamp: "Formatted at 2026-08-11 15:09:18 on dist-test-slave-1vmg"
server_key: "efca93a9dad6ef9851066fb6f7a21539"
server_key_iv: "c94ed76017847de08ccf9f670aa95ac3"
server_key_version: "encryptionkey@0"
I20260811 15:09:18.780540 14272 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "9bd694fde25346b5b538559f64d6717e"
format_stamp: "Formatted at 2026-08-11 15:09:18 on dist-test-slave-1vmg"
server_key: "efca93a9dad6ef9851066fb6f7a21539"
server_key_iv: "c94ed76017847de08ccf9f670aa95ac3"
server_key_version: "encryptionkey@0"
I20260811 15:09:18.785959 14272 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 15:09:18.788985 14294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:18.790490 14272 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:09:18.790627 14272 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "9bd694fde25346b5b538559f64d6717e"
format_stamp: "Formatted at 2026-08-11 15:09:18 on dist-test-slave-1vmg"
server_key: "efca93a9dad6ef9851066fb6f7a21539"
server_key_iv: "c94ed76017847de08ccf9f670aa95ac3"
server_key_version: "encryptionkey@0"
I20260811 15:09:18.790783 14272 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:09:18.795887 14272 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:09:18.796577 14272 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:09:18.796764 14272 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:09:18.797370 14272 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:09:18.798413 14272 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:09:18.798465 14272 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:18.798520 14272 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:09:18.798549 14272 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:18.810287 14272 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:34323
I20260811 15:09:18.810346 14475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:34323 every 8 connection(s)
I20260811 15:09:18.811622 14272 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:09:18.818698 14240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:18.813983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:54773 (local address 127.30.8.62:39573)
0811 15:09:18.814157 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:18.814161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:18.815420 (+ 1259us) server_negotiation.cc:407] Connection header received
0811 15:09:18.816885 (+ 1465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:18.816889 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:18.816941 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:18.817023 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:18.818222 (+ 1199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:09:18.818226 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:09:18.818229 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:09:18.818243 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:09:18.818264 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:09:18.818272 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:09:18.818355 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:18.818491 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 15:09:18.818548 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 15:09:18.819725 14481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:18.814384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:39573 (local address 127.30.8.1:54773)
0811 15:09:18.815257 (+ 873us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:18.815304 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:18.816645 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:18.817230 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:18.817237 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:18.817339 (+ 102us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:18.817932 (+ 593us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:09:18.817934 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:18.817980 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:09:18.818065 (+ 85us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:09:18.818343 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:09:18.818352 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:09:18.818358 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 15:09:18.818530 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 15:09:18.818800 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"thread_start_us":174,"threads_started":1}
I20260811 15:09:18.820152 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 14272
I20260811 15:09:18.820291 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:09:18.820582 30752 external_mini_cluster.cc:1428] Setting key c5e0b983f0fcc5b27b2c459cdd883f13
I20260811 15:09:18.821316 14476 heartbeater.cc:347] Connected to a master server at 127.30.8.62:39573
I20260811 15:09:18.821821 14476 heartbeater.cc:464] Registering TS with master...
I20260811 15:09:18.822602 14476 heartbeater.cc:511] Master 127.30.8.62:39573 requested a full tablet report, sending...
I20260811 15:09:18.823796 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:39573
--builtin_ntp_servers=127.30.8.20:45303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 15:09:18.825197 14176 ts_manager.cc:194] Registered new tserver with Master: 9bd694fde25346b5b538559f64d6717e (127.30.8.1:34323)
W20260811 15:09:18.958122 14482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:09:18.958426 14482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:09:18.958469 14482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:09:18.962728 14482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:09:18.962824 14482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:09:18.962908 14482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:09:18.968062 14482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:45303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:39573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14482
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:09:18.969316 14482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:09:18.970839 14482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:09:18.981755 14489 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:18.981823 14482 server_base.cc:1034] running on GCE node
W20260811 15:09:18.981755 14488 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:18.981762 14492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:18.982575 14482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:09:18.983410 14482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:09:18.984591 14482 hybrid_clock.cc:648] HybridClock initialized: now 1786460958984558 us; error 41 us; skew 500 ppm
I20260811 15:09:18.986755 14482 webserver.cc:460] Webserver started at http://127.30.8.2:44919/ using document root <none> and password file <none>
I20260811 15:09:18.987442 14482 fs_manager.cc:359] Metadata directory not provided
I20260811 15:09:18.987519 14482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:09:18.987749 14482 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:09:18.990228 14482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "b6c3216edd964c2ca17105583c728076"
format_stamp: "Formatted at 2026-08-11 15:09:18 on dist-test-slave-1vmg"
server_key: "f24b12d0c6287d7b20cc839dedf3d090"
server_key_iv: "1c25136657d39b5e7074c4a7dca1b210"
server_key_version: "encryptionkey@0"
I20260811 15:09:18.990837 14482 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "b6c3216edd964c2ca17105583c728076"
format_stamp: "Formatted at 2026-08-11 15:09:18 on dist-test-slave-1vmg"
server_key: "f24b12d0c6287d7b20cc839dedf3d090"
server_key_iv: "1c25136657d39b5e7074c4a7dca1b210"
server_key_version: "encryptionkey@0"
I20260811 15:09:18.995428 14482 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:09:18.999218 14503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:19.001040 14482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 15:09:19.001199 14482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "b6c3216edd964c2ca17105583c728076"
format_stamp: "Formatted at 2026-08-11 15:09:18 on dist-test-slave-1vmg"
server_key: "f24b12d0c6287d7b20cc839dedf3d090"
server_key_iv: "1c25136657d39b5e7074c4a7dca1b210"
server_key_version: "encryptionkey@0"
I20260811 15:09:19.001365 14482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:09:19.009166 14482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:09:19.010159 14482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:09:19.010385 14482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:09:19.011217 14482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:09:19.012606 14482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:09:19.012681 14482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:19.012733 14482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:09:19.012761 14482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:19.026140 14482 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:43095
I20260811 15:09:19.026149 14673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:43095 every 8 connection(s)
I20260811 15:09:19.027670 14482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:09:19.034976 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 14482
I20260811 15:09:19.035156 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:09:19.035492 30752 external_mini_cluster.cc:1428] Setting key d86138faec0257510ae6a9b7c7d9faba
I20260811 15:09:19.035856 14240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:19.029994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:42827 (local address 127.30.8.62:39573)
0811 15:09:19.030129 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:19.030133 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:19.031631 (+ 1498us) server_negotiation.cc:407] Connection header received
0811 15:09:19.033094 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:19.033098 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:19.033149 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:19.033229 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:19.035043 (+ 1814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:09:19.035048 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:09:19.035052 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:09:19.035070 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:09:19.035101 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:09:19.035112 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:09:19.035259 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:19.035595 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 15:09:19.035678 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 15:09:19.037066 14678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:19.030625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:39573 (local address 127.30.8.2:42827)
0811 15:09:19.031470 (+ 845us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:19.031514 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:19.032860 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:19.033452 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:19.033459 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:19.033555 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:19.034463 (+ 908us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:09:19.034466 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:19.034568 (+ 102us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:09:19.034741 (+ 173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:09:19.035330 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:09:19.035345 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:09:19.035354 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 15:09:19.035589 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 15:09:19.035926 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":391,"thread_start_us":220,"threads_started":1}
I20260811 15:09:19.038014 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:39573
--builtin_ntp_servers=127.30.8.20:45303
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 15:09:19.038559 14674 heartbeater.cc:347] Connected to a master server at 127.30.8.62:39573
I20260811 15:09:19.039011 14674 heartbeater.cc:464] Registering TS with master...
I20260811 15:09:19.039698 14674 heartbeater.cc:511] Master 127.30.8.62:39573 requested a full tablet report, sending...
I20260811 15:09:19.041501 14176 ts_manager.cc:194] Registered new tserver with Master: b6c3216edd964c2ca17105583c728076 (127.30.8.2:43095)
W20260811 15:09:19.177870 14679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:09:19.178153 14679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:09:19.178213 14679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:09:19.182564 14679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:09:19.182657 14679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:09:19.182755 14679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:09:19.188426 14679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:45303
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:39573
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14679
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:09:19.190373 14679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:09:19.192518 14679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:09:19.205281 14687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:19.205281 14692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:19.205300 14688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:19.206341 14679 server_base.cc:1034] running on GCE node
I20260811 15:09:19.207125 14679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:09:19.208039 14679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:09:19.211352 14679 hybrid_clock.cc:648] HybridClock initialized: now 1786460959211299 us; error 91 us; skew 500 ppm
I20260811 15:09:19.213717 14679 webserver.cc:460] Webserver started at http://127.30.8.3:37421/ using document root <none> and password file <none>
I20260811 15:09:19.214389 14679 fs_manager.cc:359] Metadata directory not provided
I20260811 15:09:19.214469 14679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:09:19.214697 14679 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:09:19.217106 14679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "4a4ceee2d70e416189789d7960f5af85"
format_stamp: "Formatted at 2026-08-11 15:09:19 on dist-test-slave-1vmg"
server_key: "4cf97992d45cd8a0d86c6a9afed1084f"
server_key_iv: "73985ea17f9da49367fce9ec5026b53d"
server_key_version: "encryptionkey@0"
I20260811 15:09:19.217736 14679 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "4a4ceee2d70e416189789d7960f5af85"
format_stamp: "Formatted at 2026-08-11 15:09:19 on dist-test-slave-1vmg"
server_key: "4cf97992d45cd8a0d86c6a9afed1084f"
server_key_iv: "73985ea17f9da49367fce9ec5026b53d"
server_key_version: "encryptionkey@0"
I20260811 15:09:19.222621 14679 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 15:09:19.227316 14703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:19.228940 14679 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.003s
I20260811 15:09:19.229096 14679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "4a4ceee2d70e416189789d7960f5af85"
format_stamp: "Formatted at 2026-08-11 15:09:19 on dist-test-slave-1vmg"
server_key: "4cf97992d45cd8a0d86c6a9afed1084f"
server_key_iv: "73985ea17f9da49367fce9ec5026b53d"
server_key_version: "encryptionkey@0"
I20260811 15:09:19.229238 14679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:09:19.234392 14679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:09:19.235168 14679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:09:19.235352 14679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:09:19.235960 14679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:09:19.236972 14679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:09:19.237032 14679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:19.237087 14679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:09:19.237112 14679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:19.248234 14679 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:44763
I20260811 15:09:19.248276 14884 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:44763 every 8 connection(s)
I20260811 15:09:19.249465 14679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:09:19.257082 14240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:19.252275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:32813 (local address 127.30.8.62:39573)
0811 15:09:19.252438 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:19.252443 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:19.253548 (+ 1105us) server_negotiation.cc:407] Connection header received
0811 15:09:19.255317 (+ 1769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:19.255321 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:19.255372 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:19.255457 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:19.256602 (+ 1145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:09:19.256606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:09:19.256609 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:09:19.256623 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:09:19.256643 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:09:19.256651 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:09:19.256754 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:19.256868 (+ 114us) server_negotiation.cc:299] Negotiation successful
0811 15:09:19.256921 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 15:09:19.258149 14893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:19.252447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:39573 (local address 127.30.8.3:32813)
0811 15:09:19.253362 (+ 915us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:19.253420 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:19.255064 (+ 1644us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:19.255650 (+ 586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:19.255657 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:19.255756 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:19.256330 (+ 574us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:09:19.256332 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:19.256376 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:09:19.256453 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:09:19.256722 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:09:19.256731 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:09:19.256737 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 15:09:19.256914 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 15:09:19.257185 (+ 271us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":184,"threads_started":1}
I20260811 15:09:19.258818 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 14679
I20260811 15:09:19.258939 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:09:19.259249 30752 external_mini_cluster.cc:1428] Setting key 66d353b8fe76f28af24640b0d4fb2265
I20260811 15:09:19.259402 14886 heartbeater.cc:347] Connected to a master server at 127.30.8.62:39573
I20260811 15:09:19.259671 14886 heartbeater.cc:464] Registering TS with master...
I20260811 15:09:19.260178 14886 heartbeater.cc:511] Master 127.30.8.62:39573 requested a full tablet report, sending...
I20260811 15:09:19.261377 14176 ts_manager.cc:194] Registered new tserver with Master: 4a4ceee2d70e416189789d7960f5af85 (127.30.8.3:44763)
I20260811 15:09:19.273520 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 15:09:19.276661 14240 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:09:19.275653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59604 (local address 127.30.8.62:39573)
0811 15:09:19.275795 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:19.275800 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:19.275975 (+ 175us) server_negotiation.cc:407] Connection header received
0811 15:09:19.276120 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:19.276123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:19.276165 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:19.276227 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:19.276506 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59604: BlockingRecv error: recv got EOF from 127.0.0.1:59604 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 15:09:19.276717 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 14272
I20260811 15:09:19.298868 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 14482
I20260811 15:09:19.338972 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 14679
I20260811 15:09:19.351032 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 14127
2026-08-11T15:09:19Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (1062 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:09:19 dist-test-slave-1vmg krb5kdc[14906](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:09:19 dist-test-slave-1vmg krb5kdc[14906](info): set up 2 sockets
Aug 11 15:09:19 dist-test-slave-1vmg krb5kdc[14906](info): commencing operation
krb5kdc: starting...
W20260811 15:09:20.547480 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.168s user 0.009s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:09:20 dist-test-slave-1vmg krb5kdc[14906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:09:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:09:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:09:20.695312 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:37991
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:38795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:37991
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:09:20.825366 14928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:09:20.825695 14928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:09:20.825739 14928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:09:20.830020 14928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:09:20.830122 14928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:09:20.830142 14928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:09:20.830157 14928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:09:20.830173 14928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:09:20.836077 14928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:38795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:37991
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:37991
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:09:20.837378 14928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:09:20.838714 14928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:09:20.848033 14942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:20.848089 14928 server_base.cc:1034] running on GCE node
W20260811 15:09:20.848044 14934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:20.848048 14936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:20.848891 14928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:09:20.850122 14928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:09:20.851289 14928 hybrid_clock.cc:648] HybridClock initialized: now 1786460960851262 us; error 40 us; skew 500 ppm
Aug 11 15:09:20 dist-test-slave-1vmg krb5kdc[14906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460960, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:09:20.854260 14928 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:09:20.855332 14928 webserver.cc:460] Webserver started at http://127.30.8.62:44111/ using document root <none> and password file <none>
I20260811 15:09:20.855933 14928 fs_manager.cc:359] Metadata directory not provided
I20260811 15:09:20.856004 14928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:09:20.856237 14928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:09:20.858484 14928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "1158b094d30e4cdc8cf2bf21b3d91a6f"
format_stamp: "Formatted at 2026-08-11 15:09:20 on dist-test-slave-1vmg"
server_key: "73e01e960c5b61cc0bf428bf22927af8"
server_key_iv: "a3ac0c354519d2674f23ba5b3650672c"
server_key_version: "encryptionkey@0"
I20260811 15:09:20.859131 14928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "1158b094d30e4cdc8cf2bf21b3d91a6f"
format_stamp: "Formatted at 2026-08-11 15:09:20 on dist-test-slave-1vmg"
server_key: "73e01e960c5b61cc0bf428bf22927af8"
server_key_iv: "a3ac0c354519d2674f23ba5b3650672c"
server_key_version: "encryptionkey@0"
I20260811 15:09:20.863786 14928 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 15:09:20.867062 14951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:20.868655 14928 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 15:09:20.868830 14928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "1158b094d30e4cdc8cf2bf21b3d91a6f"
format_stamp: "Formatted at 2026-08-11 15:09:20 on dist-test-slave-1vmg"
server_key: "73e01e960c5b61cc0bf428bf22927af8"
server_key_iv: "a3ac0c354519d2674f23ba5b3650672c"
server_key_version: "encryptionkey@0"
I20260811 15:09:20.868963 14928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:09:20.881883 14928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:09:20.885430 14928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:09:20.885637 14928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:09:20.893496 14928 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:37991
I20260811 15:09:20.893786 15026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:37991 every 8 connection(s)
I20260811 15:09:20.894609 14928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:09:20.895383 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 14928
I20260811 15:09:20.895496 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:09:20.895759 30752 external_mini_cluster.cc:1428] Setting key 59ca34bc26714be621de029508b850d2
I20260811 15:09:20.898027 15027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:09:20.905690 15027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f: Bootstrap starting.
I20260811 15:09:20.908315 15027 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f: Neither blocks nor log segments found. Creating new log.
Aug 11 15:09:20 dist-test-slave-1vmg krb5kdc[14906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:09:20.909324 15027 log.cc:824] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f: Log is configured to *not* fsync() on all Append() calls
I20260811 15:09:20.912261 15027 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f: No bootstrap required, opened a new log
I20260811 15:09:20.916971 15027 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1158b094d30e4cdc8cf2bf21b3d91a6f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 37991 } }
I20260811 15:09:20.917286 15027 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:09:20.917378 15027 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1158b094d30e4cdc8cf2bf21b3d91a6f, State: Initialized, Role: FOLLOWER
I20260811 15:09:20.917549 15031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:20.897264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44374 (local address 127.30.8.62:37991)
0811 15:09:20.897690 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:20.897708 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:20.897745 (+ 37us) server_negotiation.cc:407] Connection header received
0811 15:09:20.898787 (+ 1042us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:20.898833 (+ 46us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:20.899494 (+ 661us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:20.899962 (+ 468us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:20.900978 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:20.902877 (+ 1899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:20.904278 (+ 1401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:20.905722 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:20.908662 (+ 2940us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:20.910109 (+ 1447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:09:20.910117 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:09:20.910136 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:09:20.910175 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:09:20.912796 (+ 2621us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:09:20.913555 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:09:20.913565 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:09:20.913573 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:09:20.913781 (+ 208us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:09:20.914138 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:09:20.914141 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:09:20.914143 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:09:20.914818 (+ 675us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:09:20.915856 (+ 1038us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:20.916036 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 15:09:20.916457 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":265,"thread_start_us":120,"threads_started":1}
I20260811 15:09:20.917920 15027 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1158b094d30e4cdc8cf2bf21b3d91a6f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 37991 } }
I20260811 15:09:20.918078 15027 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:09:20.918143 15027 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:09:20.918253 15027 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:09:20.919529 15027 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1158b094d30e4cdc8cf2bf21b3d91a6f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 37991 } }
I20260811 15:09:20.919940 15027 leader_election.cc:302] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1158b094d30e4cdc8cf2bf21b3d91a6f; no voters:
I20260811 15:09:20.920337 15027 leader_election.cc:288] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:09:20.920433 15034 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:09:20.920665 15034 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f [term 1 LEADER]: Becoming Leader. State: Replica: 1158b094d30e4cdc8cf2bf21b3d91a6f, State: Running, Role: LEADER
I20260811 15:09:20.921010 15034 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1158b094d30e4cdc8cf2bf21b3d91a6f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 37991 } }
I20260811 15:09:20.921834 15027 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:09:20.922328 15034 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1158b094d30e4cdc8cf2bf21b3d91a6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1158b094d30e4cdc8cf2bf21b3d91a6f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 37991 } } }
I20260811 15:09:20.922454 15034 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f [sys.catalog]: This master's current role is: LEADER
I20260811 15:09:20.922857 15040 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:09:20.922878 15036 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1158b094d30e4cdc8cf2bf21b3d91a6f. Latest consensus state: current_term: 1 leader_uuid: "1158b094d30e4cdc8cf2bf21b3d91a6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1158b094d30e4cdc8cf2bf21b3d91a6f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 37991 } } }
I20260811 15:09:20.923115 15036 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f [sys.catalog]: This master's current role is: LEADER
I20260811 15:09:20.927229 15040 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:09:20.941260 15040 catalog_manager.cc:1347] Generated new cluster ID: 2a84eac363fb4063858e23fa35d01c53
I20260811 15:09:20.941380 15040 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:09:20.970618 15040 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:09:20.974295 15040 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:09:20.991242 15040 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f: Generated new TSK 0
I20260811 15:09:20.992540 15040 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
I20260811 15:09:21.039034 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:37991
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:38795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:09:21.184994 15066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:09:21.185375 15066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:09:21.185431 15066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:09:21.190491 15066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:09:21.190636 15066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:09:21.190742 15066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:09:21.197633 15066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:38795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:37991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:09:21.199082 15066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:09:21.200326 15066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:09:21.211748 15077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:21.211786 15073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:21.211786 15074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:21.212153 15066 server_base.cc:1034] running on GCE node
I20260811 15:09:21.212678 15066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:09:21.213395 15066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:09:21.214569 15066 hybrid_clock.cc:648] HybridClock initialized: now 1786460961214543 us; error 41 us; skew 500 ppm
Aug 11 15:09:21 dist-test-slave-1vmg krb5kdc[14906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460961, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:09:21.221784 15066 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:09:21.223060 15066 webserver.cc:460] Webserver started at http://127.30.8.1:37215/ using document root <none> and password file <none>
I20260811 15:09:21.223717 15066 fs_manager.cc:359] Metadata directory not provided
I20260811 15:09:21.223788 15066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:09:21.224035 15066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:09:21.226365 15066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "3e2c9eee13b74b93a4296e0e97702c07"
format_stamp: "Formatted at 2026-08-11 15:09:21 on dist-test-slave-1vmg"
server_key: "c7a0770d0548571578e8e7ff5b08f337"
server_key_iv: "bba8fbeb0003184eb44b9068e7f23c63"
server_key_version: "encryptionkey@0"
I20260811 15:09:21.227007 15066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "3e2c9eee13b74b93a4296e0e97702c07"
format_stamp: "Formatted at 2026-08-11 15:09:21 on dist-test-slave-1vmg"
server_key: "c7a0770d0548571578e8e7ff5b08f337"
server_key_iv: "bba8fbeb0003184eb44b9068e7f23c63"
server_key_version: "encryptionkey@0"
I20260811 15:09:21.231775 15066 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 15:09:21.234877 15089 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:21.236581 15066 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 15:09:21.236747 15066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "3e2c9eee13b74b93a4296e0e97702c07"
format_stamp: "Formatted at 2026-08-11 15:09:21 on dist-test-slave-1vmg"
server_key: "c7a0770d0548571578e8e7ff5b08f337"
server_key_iv: "bba8fbeb0003184eb44b9068e7f23c63"
server_key_version: "encryptionkey@0"
I20260811 15:09:21.236887 15066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:09:21.254511 15066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:09:21.257791 15066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:09:21.257997 15066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:09:21.258654 15066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:09:21.259629 15066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:09:21.259677 15066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:21.259720 15066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:09:21.259768 15066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:21.268929 15066 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:38189
I20260811 15:09:21.268953 15255 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:38189 every 8 connection(s)
I20260811 15:09:21.269955 15066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:09:21.271368 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 15066
I20260811 15:09:21.271535 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:09:21.271981 30752 external_mini_cluster.cc:1428] Setting key ed8a5d272f627d3f52c2cdd57122d91d
Aug 11 15:09:21 dist-test-slave-1vmg krb5kdc[14906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460961, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.8.2@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.2@KRBTEST.COM" created.
I20260811 15:09:21.288697 15031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:21.272728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:57081 (local address 127.30.8.62:37991)
0811 15:09:21.272934 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:21.272940 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:21.274137 (+ 1197us) server_negotiation.cc:407] Connection header received
0811 15:09:21.275937 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:21.275941 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:21.276009 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:21.276094 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:21.277810 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.278495 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.280483 (+ 1988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.280758 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.280859 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:21.284544 (+ 3685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:09:21.284547 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:09:21.284550 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:09:21.284596 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:09:21.286086 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:09:21.286885 (+ 799us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:09:21.286888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:09:21.286890 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:09:21.287004 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:09:21.287327 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:09:21.287330 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:09:21.287331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:09:21.287550 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:09:21.287650 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:21.288329 (+ 679us) server_negotiation.cc:299] Negotiation successful
0811 15:09:21.288499 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260811 15:09:21.289592 15263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:21.273174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:37991 (local address 127.30.8.1:57081)
0811 15:09:21.273925 (+ 751us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:21.274020 (+ 95us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:21.275670 (+ 1650us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:21.276356 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:21.276366 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:21.276862 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:21.277585 (+ 723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:21.277609 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.278706 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.278710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:21.280333 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:21.280349 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.280963 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.280969 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:21.281228 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:21.282045 (+ 817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:21.282074 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:21.284275 (+ 2201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:09:21.286340 (+ 2065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:09:21.286349 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:09:21.286371 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:09:21.286767 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:09:21.287077 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:09:21.287079 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:09:21.287080 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:09:21.287245 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:09:21.287665 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:09:21.287672 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:09:21.288007 (+ 335us) client_negotiation.cc:769] Sending connection context
0811 15:09:21.288216 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 15:09:21.288600 (+ 384us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":123,"threads_started":1}
I20260811 15:09:21.291100 15256 heartbeater.cc:347] Connected to a master server at 127.30.8.62:37991
I20260811 15:09:21.291373 15256 heartbeater.cc:464] Registering TS with master...
I20260811 15:09:21.291932 15256 heartbeater.cc:511] Master 127.30.8.62:37991 requested a full tablet report, sending...
I20260811 15:09:21.293901 14975 ts_manager.cc:194] Registered new tserver with Master: 3e2c9eee13b74b93a4296e0e97702c07 (127.30.8.1:38189)
I20260811 15:09:21.295691 14975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:57081
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
I20260811 15:09:21.310242 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:37991
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:38795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:09:21.449108 15269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:09:21.449435 15269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:09:21.449481 15269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:09:21.453728 15269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:09:21.453828 15269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:09:21.453897 15269 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:09:21.459702 15269 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:38795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:37991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:09:21.460951 15269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:09:21.462030 15269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:09:21.474637 15277 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:21.474658 15279 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:21.474637 15276 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:21.475021 15269 server_base.cc:1034] running on GCE node
I20260811 15:09:21.475709 15269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:09:21.476495 15269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:09:21.477675 15269 hybrid_clock.cc:648] HybridClock initialized: now 1786460961477656 us; error 40 us; skew 500 ppm
Aug 11 15:09:21 dist-test-slave-1vmg krb5kdc[14906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460961, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:09:21.481037 15269 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:09:21.482141 15269 webserver.cc:460] Webserver started at http://127.30.8.2:36893/ using document root <none> and password file <none>
I20260811 15:09:21.482779 15269 fs_manager.cc:359] Metadata directory not provided
I20260811 15:09:21.482853 15269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:09:21.483119 15269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:09:21.485383 15269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "096ffc379c9041e58fee5169f5739c24"
format_stamp: "Formatted at 2026-08-11 15:09:21 on dist-test-slave-1vmg"
server_key: "352ca6908d9e781dbcf843f7dacb7238"
server_key_iv: "dd11b28ffe62f67c383d6849e60a3d27"
server_key_version: "encryptionkey@0"
I20260811 15:09:21.485956 15269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "096ffc379c9041e58fee5169f5739c24"
format_stamp: "Formatted at 2026-08-11 15:09:21 on dist-test-slave-1vmg"
server_key: "352ca6908d9e781dbcf843f7dacb7238"
server_key_iv: "dd11b28ffe62f67c383d6849e60a3d27"
server_key_version: "encryptionkey@0"
I20260811 15:09:21.490186 15269 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 15:09:21.493001 15288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:21.494419 15269 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 15:09:21.494553 15269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "096ffc379c9041e58fee5169f5739c24"
format_stamp: "Formatted at 2026-08-11 15:09:21 on dist-test-slave-1vmg"
server_key: "352ca6908d9e781dbcf843f7dacb7238"
server_key_iv: "dd11b28ffe62f67c383d6849e60a3d27"
server_key_version: "encryptionkey@0"
I20260811 15:09:21.494673 15269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:09:21.511647 15269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:09:21.515120 15269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:09:21.515326 15269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:09:21.515982 15269 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:09:21.516908 15269 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:09:21.516960 15269 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:21.517006 15269 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:09:21.517036 15269 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:21.526417 15269 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:42215
I20260811 15:09:21.526443 15461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:42215 every 8 connection(s)
I20260811 15:09:21.527460 15269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:09:21.531569 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 15269
I20260811 15:09:21.531677 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:09:21.531944 30752 external_mini_cluster.cc:1428] Setting key 1f068cbaa7b4523796d269ddf0e15812
Aug 11 15:09:21 dist-test-slave-1vmg krb5kdc[14906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460961, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.8.3@KRBTEST.COM; defaulting to no policy
I20260811 15:09:21.543313 15031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:21.529649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:39473 (local address 127.30.8.62:37991)
0811 15:09:21.529804 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:21.529809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:21.530744 (+ 935us) server_negotiation.cc:407] Connection header received
0811 15:09:21.532232 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:21.532236 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:21.532300 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:21.532381 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:21.533941 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.534644 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.535930 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.536152 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.536243 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:21.539282 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:09:21.539286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:09:21.539288 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:09:21.539315 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:09:21.540745 (+ 1430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:09:21.541430 (+ 685us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:09:21.541433 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:09:21.541434 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:09:21.541515 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:09:21.541969 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:09:21.541972 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:09:21.541973 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:09:21.542180 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:09:21.542263 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:21.543028 (+ 765us) server_negotiation.cc:299] Negotiation successful
0811 15:09:21.543165 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 15:09:21.544283 15464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:21.530053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:37991 (local address 127.30.8.2:39473)
0811 15:09:21.530567 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:21.530619 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:21.531995 (+ 1376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:21.532595 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:21.532602 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:21.533113 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:21.533778 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:21.533795 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.534776 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.534779 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:21.535798 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:21.535809 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.536319 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.536323 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:21.536489 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:21.537081 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:21.537100 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:21.539066 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:09:21.540887 (+ 1821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:09:21.540895 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:09:21.540916 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:09:21.541302 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:09:21.541601 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:09:21.541604 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:09:21.541606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:09:21.541864 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:09:21.542294 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:09:21.542302 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:09:21.542719 (+ 417us) client_negotiation.cc:769] Sending connection context
0811 15:09:21.542979 (+ 260us) client_negotiation.cc:240] Negotiation successful
0811 15:09:21.543391 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":131,"threads_started":1}
I20260811 15:09:21.545673 15462 heartbeater.cc:347] Connected to a master server at 127.30.8.62:37991
I20260811 15:09:21.546129 15462 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.3@KRBTEST.COM" created.
I20260811 15:09:21.547061 15462 heartbeater.cc:511] Master 127.30.8.62:37991 requested a full tablet report, sending...
I20260811 15:09:21.548671 14975 ts_manager.cc:194] Registered new tserver with Master: 096ffc379c9041e58fee5169f5739c24 (127.30.8.2:42215)
I20260811 15:09:21.549335 14975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:39473
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
I20260811 15:09:21.567878 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:37991
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:38795
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:09:21.694376 15469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:09:21.694772 15469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:09:21.694823 15469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:09:21.699373 15469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:09:21.699469 15469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:09:21.699558 15469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:09:21.705874 15469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:38795
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:37991
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:09:21.707433 15469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:09:21.708781 15469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:09:21.722201 15481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:09:21.722254 15483 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:21.722535 15469 server_base.cc:1034] running on GCE node
W20260811 15:09:21.722575 15488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:09:21.723245 15469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:09:21.724102 15469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:09:21.725278 15469 hybrid_clock.cc:648] HybridClock initialized: now 1786460961725244 us; error 65 us; skew 500 ppm
Aug 11 15:09:21 dist-test-slave-1vmg krb5kdc[14906](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460961, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:09:21.728448 15469 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:09:21.729513 15469 webserver.cc:460] Webserver started at http://127.30.8.3:46557/ using document root <none> and password file <none>
I20260811 15:09:21.730126 15469 fs_manager.cc:359] Metadata directory not provided
I20260811 15:09:21.730191 15469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:09:21.730386 15469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:09:21.732653 15469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "c372a5f49306486981234eb514a505a8"
format_stamp: "Formatted at 2026-08-11 15:09:21 on dist-test-slave-1vmg"
server_key: "d9f23cea0f4a8c996d6d20c7c4f5dd64"
server_key_iv: "a5a15166bc63b9c1bf87e75aaa0da273"
server_key_version: "encryptionkey@0"
I20260811 15:09:21.733285 15469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "c372a5f49306486981234eb514a505a8"
format_stamp: "Formatted at 2026-08-11 15:09:21 on dist-test-slave-1vmg"
server_key: "d9f23cea0f4a8c996d6d20c7c4f5dd64"
server_key_iv: "a5a15166bc63b9c1bf87e75aaa0da273"
server_key_version: "encryptionkey@0"
I20260811 15:09:21.737360 15469 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 15:09:21.740533 15497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:21.742205 15469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:09:21.742372 15469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "c372a5f49306486981234eb514a505a8"
format_stamp: "Formatted at 2026-08-11 15:09:21 on dist-test-slave-1vmg"
server_key: "d9f23cea0f4a8c996d6d20c7c4f5dd64"
server_key_iv: "a5a15166bc63b9c1bf87e75aaa0da273"
server_key_version: "encryptionkey@0"
I20260811 15:09:21.742509 15469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:09:21.761390 15469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:09:21.764887 15469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:09:21.765121 15469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:09:21.765779 15469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:09:21.766690 15469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:09:21.766741 15469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:21.766784 15469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:09:21.766804 15469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:09:21.776335 15469 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:43605
I20260811 15:09:21.776361 15676 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:43605 every 8 connection(s)
I20260811 15:09:21.777333 15469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:09:21.778080 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 15469
I20260811 15:09:21.778208 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:09:21.778568 30752 external_mini_cluster.cc:1428] Setting key f3d816c02560a6b347470aedeedff74e
Aug 11 15:09:21 dist-test-slave-1vmg krb5kdc[14906](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460961, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:09:21.792029 15031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:21.779390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:46695 (local address 127.30.8.62:37991)
0811 15:09:21.779517 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:21.779521 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:21.780752 (+ 1231us) server_negotiation.cc:407] Connection header received
0811 15:09:21.782265 (+ 1513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:21.782268 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:21.782325 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:21.782399 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:21.783750 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.784287 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.785477 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.785711 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.785802 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:21.788492 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:09:21.788496 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:09:21.788499 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:09:21.788528 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:09:21.789817 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:09:21.790341 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:09:21.790345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:09:21.790347 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:09:21.790430 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:09:21.790770 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:09:21.790774 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:09:21.790776 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:09:21.791030 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:09:21.791148 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:21.791746 (+ 598us) server_negotiation.cc:299] Negotiation successful
0811 15:09:21.791889 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 15:09:21.793149 15680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:21.780036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:37991 (local address 127.30.8.3:46695)
0811 15:09:21.780608 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:21.780655 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:21.782037 (+ 1382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:21.782591 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:21.782598 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:21.783061 (+ 463us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:09:21.783604 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:21.783618 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.784419 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.784422 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:21.785351 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:21.785363 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.785866 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.785870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:21.786056 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:21.786674 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:09:21.786692 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:09:21.788307 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:09:21.789934 (+ 1627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:09:21.789941 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:09:21.789961 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:09:21.790241 (+ 280us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:09:21.790517 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:09:21.790520 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:09:21.790522 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:09:21.790676 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:09:21.791133 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:09:21.791141 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:09:21.791435 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 15:09:21.791670 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 15:09:21.792026 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"thread_start_us":169,"threads_started":1}
I20260811 15:09:21.794301 15677 heartbeater.cc:347] Connected to a master server at 127.30.8.62:37991
I20260811 15:09:21.794559 15677 heartbeater.cc:464] Registering TS with master...
I20260811 15:09:21.795143 15677 heartbeater.cc:511] Master 127.30.8.62:37991 requested a full tablet report, sending...
I20260811 15:09:21.796311 14975 ts_manager.cc:194] Registered new tserver with Master: c372a5f49306486981234eb514a505a8 (127.30.8.3:43605)
I20260811 15:09:21.796902 14975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:46695
I20260811 15:09:21.806275 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 15:09:21.815305 15031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:21.808637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44382 (local address 127.30.8.62:37991)
0811 15:09:21.808764 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:21.808767 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:21.808850 (+ 83us) server_negotiation.cc:407] Connection header received
0811 15:09:21.808962 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:21.808965 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:21.809010 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:21.809105 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:09:21.809781 (+ 676us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.810308 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.811140 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.811344 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.811415 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:21.812691 (+ 1276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:09:21.812694 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:09:21.812696 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:09:21.812721 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:09:21.813841 (+ 1120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:09:21.814224 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:09:21.814227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:09:21.814228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:09:21.814313 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:09:21.814520 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:09:21.814522 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:09:21.814523 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:09:21.814756 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:09:21.814861 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:21.815070 (+ 209us) server_negotiation.cc:299] Negotiation successful
0811 15:09:21.815172 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 15:09:21.818257 14975 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:09:21.821588 14975 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:09:21.838764 15710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:21.833549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46496 (local address 127.30.8.3:43605)
0811 15:09:21.834170 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:21.834182 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:21.834203 (+ 21us) server_negotiation.cc:407] Connection header received
0811 15:09:21.834273 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:21.834278 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:21.834457 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:21.834877 (+ 420us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:09:21.835056 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.835786 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.837338 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.838069 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.838169 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:21.838337 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:21.838494 (+ 157us) server_negotiation.cc:299] Negotiation successful
0811 15:09:21.838603 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":520,"thread_start_us":130,"threads_started":1}
I20260811 15:09:21.838761 15703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:21.832750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:43605 (local address 127.0.0.1:46496)
0811 15:09:21.833145 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:21.833159 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:21.833309 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:21.834605 (+ 1296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:21.834608 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:21.834626 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:09:21.834746 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:21.834753 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.835916 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.835919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:21.837191 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:21.837203 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.838187 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.838191 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:21.838297 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:21.838335 (+ 38us) client_negotiation.cc:769] Sending connection context
0811 15:09:21.838451 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 15:09:21.838592 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"spinlock_wait_cycles":2048,"thread_start_us":108,"threads_started":1}
I20260811 15:09:21.839082 15697 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:21.832705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:42215 (local address 127.0.0.1:42940)
0811 15:09:21.833102 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:21.833159 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:21.833309 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:21.834508 (+ 1199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:21.834514 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:21.834537 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:09:21.834724 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:21.834732 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.836411 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.836413 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:21.837563 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:21.837574 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.838657 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.838661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:21.838749 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:21.838771 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 15:09:21.838822 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 15:09:21.838893 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":102,"spinlock_wait_cycles":4992,"thread_start_us":105,"threads_started":1}
I20260811 15:09:21.839406 15711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:21.833956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56854 (local address 127.30.8.1:38189)
0811 15:09:21.834303 (+ 347us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:21.834309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:21.834330 (+ 21us) server_negotiation.cc:407] Connection header received
0811 15:09:21.834404 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:21.834409 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:21.834592 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:21.834737 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:09:21.835146 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.836264 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.837723 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.838507 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.838623 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:21.838763 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:21.839161 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 15:09:21.839266 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":90,"threads_started":1}
I20260811 15:09:21.839681 15705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:21.833500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:38189 (local address 127.0.0.1:56854)
0811 15:09:21.833783 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:21.833796 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:21.833883 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:21.834882 (+ 999us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:21.834884 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:21.834900 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:09:21.835017 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:21.835024 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.836395 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.836398 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:21.837620 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:21.837631 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.838821 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.838824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:21.838897 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:21.838950 (+ 53us) client_negotiation.cc:769] Sending connection context
0811 15:09:21.839011 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 15:09:21.839082 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":51,"thread_start_us":137,"threads_started":1}
I20260811 15:09:21.839838 15707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:21.833479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42940 (local address 127.30.8.2:42215)
0811 15:09:21.833947 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:21.833960 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:21.833987 (+ 27us) server_negotiation.cc:407] Connection header received
0811 15:09:21.834098 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:21.834104 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:21.834324 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:21.834876 (+ 552us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:09:21.835067 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.836262 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.837688 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:21.838507 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:21.839358 (+ 851us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:21.839510 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:21.839574 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 15:09:21.839672 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":146,"threads_started":1}
I20260811 15:09:21.842576 15360 tablet_service.cc:1467] Processing CreateTablet for tablet a76d6c0ff8f04c4b98c997f63dcefc31 (DEFAULT_TABLE table=test-table [id=bc011cbbcd504222a9b1540644d77cf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:09:21.842576 15571 tablet_service.cc:1467] Processing CreateTablet for tablet a76d6c0ff8f04c4b98c997f63dcefc31 (DEFAULT_TABLE table=test-table [id=bc011cbbcd504222a9b1540644d77cf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:09:21.842574 15151 tablet_service.cc:1467] Processing CreateTablet for tablet a76d6c0ff8f04c4b98c997f63dcefc31 (DEFAULT_TABLE table=test-table [id=bc011cbbcd504222a9b1540644d77cf2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:09:21.843786 15360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a76d6c0ff8f04c4b98c997f63dcefc31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:09:21.844439 15151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a76d6c0ff8f04c4b98c997f63dcefc31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:09:21.844441 15571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a76d6c0ff8f04c4b98c997f63dcefc31. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:09:21.851289 15714 tablet_bootstrap.cc:492] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24: Bootstrap starting.
I20260811 15:09:21.853780 15714 tablet_bootstrap.cc:654] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24: Neither blocks nor log segments found. Creating new log.
I20260811 15:09:21.854008 15717 tablet_bootstrap.cc:492] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8: Bootstrap starting.
I20260811 15:09:21.855350 15716 tablet_bootstrap.cc:492] T a76d6c0ff8f04c4b98c997f63dcefc31 P 3e2c9eee13b74b93a4296e0e97702c07: Bootstrap starting.
I20260811 15:09:21.855448 15714 log.cc:824] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24: Log is configured to *not* fsync() on all Append() calls
I20260811 15:09:21.856788 15717 tablet_bootstrap.cc:654] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8: Neither blocks nor log segments found. Creating new log.
I20260811 15:09:21.858011 15714 tablet_bootstrap.cc:492] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24: No bootstrap required, opened a new log
I20260811 15:09:21.858111 15716 tablet_bootstrap.cc:654] T a76d6c0ff8f04c4b98c997f63dcefc31 P 3e2c9eee13b74b93a4296e0e97702c07: Neither blocks nor log segments found. Creating new log.
I20260811 15:09:21.858148 15717 log.cc:824] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8: Log is configured to *not* fsync() on all Append() calls
I20260811 15:09:21.858249 15714 ts_tablet_manager.cc:1397] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 15:09:21.859136 15716 log.cc:824] T a76d6c0ff8f04c4b98c997f63dcefc31 P 3e2c9eee13b74b93a4296e0e97702c07: Log is configured to *not* fsync() on all Append() calls
I20260811 15:09:21.860888 15717 tablet_bootstrap.cc:492] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8: No bootstrap required, opened a new log
I20260811 15:09:21.861141 15717 ts_tablet_manager.cc:1397] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 15:09:21.861596 15716 tablet_bootstrap.cc:492] T a76d6c0ff8f04c4b98c997f63dcefc31 P 3e2c9eee13b74b93a4296e0e97702c07: No bootstrap required, opened a new log
I20260811 15:09:21.861829 15716 ts_tablet_manager.cc:1397] T a76d6c0ff8f04c4b98c997f63dcefc31 P 3e2c9eee13b74b93a4296e0e97702c07: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 15:09:21.862306 15714 raft_consensus.cc:348] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "096ffc379c9041e58fee5169f5739c24" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42215 } } peers { permanent_uuid: "c372a5f49306486981234eb514a505a8" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43605 } } peers { permanent_uuid: "3e2c9eee13b74b93a4296e0e97702c07" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 38189 } }
I20260811 15:09:21.862625 15714 raft_consensus.cc:374] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:09:21.862725 15714 raft_consensus.cc:729] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 096ffc379c9041e58fee5169f5739c24, State: Initialized, Role: FOLLOWER
I20260811 15:09:21.863338 15714 consensus_queue.cc:260] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "096ffc379c9041e58fee5169f5739c24" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42215 } } peers { permanent_uuid: "c372a5f49306486981234eb514a505a8" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43605 } } peers { permanent_uuid: "3e2c9eee13b74b93a4296e0e97702c07" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 38189 } }
I20260811 15:09:21.864753 15714 ts_tablet_manager.cc:1428] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 15:09:21.865203 15462 heartbeater.cc:503] Master 127.30.8.62:37991 was elected leader, sending a full tablet report...
I20260811 15:09:21.866297 15717 raft_consensus.cc:348] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "096ffc379c9041e58fee5169f5739c24" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42215 } } peers { permanent_uuid: "c372a5f49306486981234eb514a505a8" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43605 } } peers { permanent_uuid: "3e2c9eee13b74b93a4296e0e97702c07" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 38189 } }
I20260811 15:09:21.866568 15717 raft_consensus.cc:374] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:09:21.866331 15716 raft_consensus.cc:348] T a76d6c0ff8f04c4b98c997f63dcefc31 P 3e2c9eee13b74b93a4296e0e97702c07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "096ffc379c9041e58fee5169f5739c24" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42215 } } peers { permanent_uuid: "c372a5f49306486981234eb514a505a8" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43605 } } peers { permanent_uuid: "3e2c9eee13b74b93a4296e0e97702c07" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 38189 } }
I20260811 15:09:21.866660 15717 raft_consensus.cc:729] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c372a5f49306486981234eb514a505a8, State: Initialized, Role: FOLLOWER
I20260811 15:09:21.866765 15716 raft_consensus.cc:374] T a76d6c0ff8f04c4b98c997f63dcefc31 P 3e2c9eee13b74b93a4296e0e97702c07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:09:21.866852 15716 raft_consensus.cc:729] T a76d6c0ff8f04c4b98c997f63dcefc31 P 3e2c9eee13b74b93a4296e0e97702c07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e2c9eee13b74b93a4296e0e97702c07, State: Initialized, Role: FOLLOWER
I20260811 15:09:21.867198 15717 consensus_queue.cc:260] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "096ffc379c9041e58fee5169f5739c24" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42215 } } peers { permanent_uuid: "c372a5f49306486981234eb514a505a8" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43605 } } peers { permanent_uuid: "3e2c9eee13b74b93a4296e0e97702c07" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 38189 } }
I20260811 15:09:21.867422 15716 consensus_queue.cc:260] T a76d6c0ff8f04c4b98c997f63dcefc31 P 3e2c9eee13b74b93a4296e0e97702c07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "096ffc379c9041e58fee5169f5739c24" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42215 } } peers { permanent_uuid: "c372a5f49306486981234eb514a505a8" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43605 } } peers { permanent_uuid: "3e2c9eee13b74b93a4296e0e97702c07" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 38189 } }
I20260811 15:09:21.867956 15677 heartbeater.cc:503] Master 127.30.8.62:37991 was elected leader, sending a full tablet report...
I20260811 15:09:21.868326 15717 ts_tablet_manager.cc:1428] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 15:09:21.868736 15716 ts_tablet_manager.cc:1428] T a76d6c0ff8f04c4b98c997f63dcefc31 P 3e2c9eee13b74b93a4296e0e97702c07: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 15:09:21.869154 15256 heartbeater.cc:503] Master 127.30.8.62:37991 was elected leader, sending a full tablet report...
W20260811 15:09:22.021512 15260 tablet.cc:2369] T a76d6c0ff8f04c4b98c997f63dcefc31 P 3e2c9eee13b74b93a4296e0e97702c07: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 15:09:22.028424 15679 tablet.cc:2369] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 15:09:22.028714 15463 tablet.cc:2369] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:09:22.072105 15723 raft_consensus.cc:482] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:09:22.072454 15723 raft_consensus.cc:504] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "096ffc379c9041e58fee5169f5739c24" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42215 } } peers { permanent_uuid: "c372a5f49306486981234eb514a505a8" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43605 } } peers { permanent_uuid: "3e2c9eee13b74b93a4296e0e97702c07" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 38189 } }
I20260811 15:09:22.074159 15723 leader_election.cc:288] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c372a5f49306486981234eb514a505a8 (127.30.8.3:43605), 3e2c9eee13b74b93a4296e0e97702c07 (127.30.8.1:38189)
I20260811 15:09:22.079371 15711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:22.075284 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:55993 (local address 127.30.8.1:38189)
0811 15:09:22.075490 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:22.075496 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:22.075513 (+ 17us) server_negotiation.cc:407] Connection header received
0811 15:09:22.075574 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:22.075576 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:22.075621 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:22.075688 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:09:22.076002 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:22.076558 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:22.077842 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:22.078503 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:22.078584 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:22.078672 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:22.079094 (+ 422us) server_negotiation.cc:299] Negotiation successful
0811 15:09:22.079198 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 15:09:22.079371 15710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:22.074659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:36947 (local address 127.30.8.3:43605)
0811 15:09:22.074823 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:22.074826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:22.075027 (+ 201us) server_negotiation.cc:407] Connection header received
0811 15:09:22.075248 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:22.075252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:22.075322 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:22.075441 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:09:22.075828 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:22.076367 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:22.077735 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:22.078453 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:22.078584 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:22.078671 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:22.079094 (+ 423us) server_negotiation.cc:299] Negotiation successful
0811 15:09:22.079198 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 15:09:22.079407 15726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:22.074510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:43605 (local address 127.30.8.2:36947)
0811 15:09:22.074862 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:22.074879 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:22.075080 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:22.075523 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:22.075528 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:22.075561 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:09:22.075688 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:22.075695 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:22.076486 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:22.076489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:22.077550 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:22.077564 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:22.078628 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:22.078632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:22.078856 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:22.078896 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 15:09:22.079074 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 15:09:22.079187 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":256,"mutex_wait_us":1,"thread_start_us":135,"threads_started":1}
I20260811 15:09:22.080075 15727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:22.075069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:38189 (local address 127.30.8.2:55993)
0811 15:09:22.075314 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:22.075326 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 15:09:22.075434 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:09:22.075718 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:09:22.075722 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:09:22.075742 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:09:22.075885 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:22.075891 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:22.076668 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:22.076672 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:22.077686 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:09:22.077699 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:09:22.078772 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:09:22.078774 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:09:22.078895 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:22.078961 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 15:09:22.079804 (+ 843us) client_negotiation.cc:240] Negotiation successful
0811 15:09:22.079911 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":95,"threads_started":1}
I20260811 15:09:22.080492 15608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a76d6c0ff8f04c4b98c997f63dcefc31" candidate_uuid: "096ffc379c9041e58fee5169f5739c24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372a5f49306486981234eb514a505a8" is_pre_election: true
I20260811 15:09:22.080997 15608 raft_consensus.cc:2454] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 096ffc379c9041e58fee5169f5739c24 in term 0.
I20260811 15:09:22.081174 15183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a76d6c0ff8f04c4b98c997f63dcefc31" candidate_uuid: "096ffc379c9041e58fee5169f5739c24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e2c9eee13b74b93a4296e0e97702c07" is_pre_election: true
I20260811 15:09:22.081492 15183 raft_consensus.cc:2454] T a76d6c0ff8f04c4b98c997f63dcefc31 P 3e2c9eee13b74b93a4296e0e97702c07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 096ffc379c9041e58fee5169f5739c24 in term 0.
I20260811 15:09:22.081676 15290 leader_election.cc:302] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 096ffc379c9041e58fee5169f5739c24, c372a5f49306486981234eb514a505a8; no voters:
I20260811 15:09:22.082083 15723 raft_consensus.cc:2781] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:09:22.082185 15723 raft_consensus.cc:482] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:09:22.082238 15723 raft_consensus.cc:3037] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:09:22.083521 15723 raft_consensus.cc:504] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "096ffc379c9041e58fee5169f5739c24" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42215 } } peers { permanent_uuid: "c372a5f49306486981234eb514a505a8" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43605 } } peers { permanent_uuid: "3e2c9eee13b74b93a4296e0e97702c07" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 38189 } }
I20260811 15:09:22.083928 15723 leader_election.cc:288] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [CANDIDATE]: Term 1 election: Requested vote from peers c372a5f49306486981234eb514a505a8 (127.30.8.3:43605), 3e2c9eee13b74b93a4296e0e97702c07 (127.30.8.1:38189)
I20260811 15:09:22.084491 15608 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a76d6c0ff8f04c4b98c997f63dcefc31" candidate_uuid: "096ffc379c9041e58fee5169f5739c24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c372a5f49306486981234eb514a505a8"
I20260811 15:09:22.084543 15183 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a76d6c0ff8f04c4b98c997f63dcefc31" candidate_uuid: "096ffc379c9041e58fee5169f5739c24" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e2c9eee13b74b93a4296e0e97702c07"
I20260811 15:09:22.084668 15608 raft_consensus.cc:3037] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:09:22.084698 15183 raft_consensus.cc:3037] T a76d6c0ff8f04c4b98c997f63dcefc31 P 3e2c9eee13b74b93a4296e0e97702c07 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:09:22.085899 15608 raft_consensus.cc:2454] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 096ffc379c9041e58fee5169f5739c24 in term 1.
I20260811 15:09:22.085930 15183 raft_consensus.cc:2454] T a76d6c0ff8f04c4b98c997f63dcefc31 P 3e2c9eee13b74b93a4296e0e97702c07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 096ffc379c9041e58fee5169f5739c24 in term 1.
I20260811 15:09:22.086238 15290 leader_election.cc:302] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 096ffc379c9041e58fee5169f5739c24, 3e2c9eee13b74b93a4296e0e97702c07; no voters:
I20260811 15:09:22.086422 15723 raft_consensus.cc:2781] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:09:22.086694 15723 raft_consensus.cc:686] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [term 1 LEADER]: Becoming Leader. State: Replica: 096ffc379c9041e58fee5169f5739c24, State: Running, Role: LEADER
I20260811 15:09:22.087126 15723 consensus_queue.cc:237] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "096ffc379c9041e58fee5169f5739c24" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42215 } } peers { permanent_uuid: "c372a5f49306486981234eb514a505a8" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43605 } } peers { permanent_uuid: "3e2c9eee13b74b93a4296e0e97702c07" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 38189 } }
I20260811 15:09:22.090416 14967 catalog_manager.cc:5495] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 reported cstate change: term changed from 0 to 1, leader changed from <none> to 096ffc379c9041e58fee5169f5739c24 (127.30.8.2). New cstate: current_term: 1 leader_uuid: "096ffc379c9041e58fee5169f5739c24" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "096ffc379c9041e58fee5169f5739c24" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42215 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c372a5f49306486981234eb514a505a8" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43605 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e2c9eee13b74b93a4296e0e97702c07" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 38189 } health_report { overall_health: UNKNOWN } } }
I20260811 15:09:22.136427 15707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:22.133021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42950 (local address 127.30.8.2:42215)
0811 15:09:22.133174 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 15:09:22.133179 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:09:22.133193 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:09:22.133321 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:09:22.133324 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:09:22.133381 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:09:22.133474 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:09:22.133685 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:22.134186 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:22.135017 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:09:22.135223 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:09:22.135297 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:09:22.135554 (+ 257us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:09:22.136085 (+ 531us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:09:22.136182 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 15:09:22.136212 (+ 30us) server_negotiation.cc:299] Negotiation successful
0811 15:09:22.136268 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 15:09:22.143818 15183 raft_consensus.cc:1261] T a76d6c0ff8f04c4b98c997f63dcefc31 P 3e2c9eee13b74b93a4296e0e97702c07 [term 1 FOLLOWER]: Refusing update from remote peer 096ffc379c9041e58fee5169f5739c24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:09:22.144028 15608 raft_consensus.cc:1261] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8 [term 1 FOLLOWER]: Refusing update from remote peer 096ffc379c9041e58fee5169f5739c24: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:09:22.144790 15723 consensus_queue.cc:1035] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e2c9eee13b74b93a4296e0e97702c07" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 38189 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:09:22.145087 15728 consensus_queue.cc:1035] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c372a5f49306486981234eb514a505a8" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43605 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:09:22.155372 15738 mvcc.cc:205] Tried to move back new op lower bound from 7317344100931661824 to 7317344100711530496. Current Snapshot: MvccSnapshot[applied={T|T < 7317344100931661824}]
I20260811 15:09:22.156628 15737 mvcc.cc:205] Tried to move back new op lower bound from 7317344100931661824 to 7317344100711530496. Current Snapshot: MvccSnapshot[applied={T|T < 7317344100931661824}]
I20260811 15:09:22.164129 14967 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:09:22.164330 14967 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:09:22.166751 14967 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1158b094d30e4cdc8cf2bf21b3d91a6f: Sending DeleteTablet for 3 replicas of tablet a76d6c0ff8f04c4b98c997f63dcefc31
I20260811 15:09:22.167879 15151 tablet_service.cc:1514] Processing DeleteTablet for tablet a76d6c0ff8f04c4b98c997f63dcefc31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:09:22 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:56854
I20260811 15:09:22.168052 15571 tablet_service.cc:1514] Processing DeleteTablet for tablet a76d6c0ff8f04c4b98c997f63dcefc31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:09:22 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:46496
I20260811 15:09:22.168087 15360 tablet_service.cc:1514] Processing DeleteTablet for tablet a76d6c0ff8f04c4b98c997f63dcefc31 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:09:22 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:42940
I20260811 15:09:22.168893 15754 tablet_replica.cc:330] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8: stopping tablet replica
I20260811 15:09:22.168963 15755 tablet_replica.cc:330] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24: stopping tablet replica
I20260811 15:09:22.169312 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 15066
I20260811 15:09:22.169392 15754 raft_consensus.cc:2229] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:09:22.170331 15754 raft_consensus.cc:2258] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:09:22.172931 15755 raft_consensus.cc:2229] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:09:22.173537 15755 raft_consensus.cc:2258] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:09:22.174633 15754 ts_tablet_manager.cc:1905] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:09:22.175818 15755 ts_tablet_manager.cc:1905] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:09:22.178540 15754 ts_tablet_manager.cc:1918] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:09:22.178719 15754 log.cc:1196] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/a76d6c0ff8f04c4b98c997f63dcefc31
I20260811 15:09:22.179198 15754 ts_tablet_manager.cc:1939] T a76d6c0ff8f04c4b98c997f63dcefc31 P c372a5f49306486981234eb514a505a8: Deleting consensus metadata
I20260811 15:09:22.181144 14953 catalog_manager.cc:4841] TS c372a5f49306486981234eb514a505a8 (127.30.8.3:43605): tablet a76d6c0ff8f04c4b98c997f63dcefc31 (table test-table [id=bc011cbbcd504222a9b1540644d77cf2]) successfully deleted
I20260811 15:09:22.181793 15755 ts_tablet_manager.cc:1918] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:09:22.181999 15755 log.cc:1196] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/a76d6c0ff8f04c4b98c997f63dcefc31
I20260811 15:09:22.182404 15755 ts_tablet_manager.cc:1939] T a76d6c0ff8f04c4b98c997f63dcefc31 P 096ffc379c9041e58fee5169f5739c24: Deleting consensus metadata
I20260811 15:09:22.184829 14952 catalog_manager.cc:4841] TS 096ffc379c9041e58fee5169f5739c24 (127.30.8.2:42215): tablet a76d6c0ff8f04c4b98c997f63dcefc31 (table test-table [id=bc011cbbcd504222a9b1540644d77cf2]) successfully deleted
W20260811 15:09:22.190171 14953 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.8.1:38189 (error 108)
I20260811 15:09:22.190457 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 15269
I20260811 15:09:22.199478 15705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:09:22.199112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:38189 (local address 127.0.0.1:56864)
0811 15:09:22.199266 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 15:09:22.199355 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.1:38189: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 15:09:22.199717 14953 catalog_manager.cc:4584] TS 3e2c9eee13b74b93a4296e0e97702c07 (127.30.8.1:38189): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a76d6c0ff8f04c4b98c997f63dcefc31: Network error: Client connection negotiation failed: client connection to 127.30.8.1:38189: connect: Connection refused (error 111)
I20260811 15:09:22.206432 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 15469
I20260811 15:09:22.231992 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 14928
2026-08-11T15:09:22Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2883 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:09:22 dist-test-slave-1vmg krb5kdc[15762](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:09:22 dist-test-slave-1vmg krb5kdc[15762](info): set up 2 sockets
Aug 11 15:09:22 dist-test-slave-1vmg krb5kdc[15762](info): commencing operation
krb5kdc: starting...
W20260811 15:09:23.447332 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.185s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:09:23 dist-test-slave-1vmg krb5kdc[15762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460963, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:09:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:09:23Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.8.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.8.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.8.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/rangeradmin_127.30.8.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.8.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/rangeradmin_127.30.8.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.8.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.8.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.8.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/rangerlookup_127.30.8.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.8.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/rangerlookup_127.30.8.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.8.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.8.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.8.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/HTTP_127.30.8.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.8.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/HTTP_127.30.8.20@KRBTEST.COM.keytab.
I20260811 15:09:23.653563 30752 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskxgyPSN/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/postgres -l logfile start
2026-08-11 15:09:26.254 UTC [15808] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 15:09:26.254 UTC [15808] LOG: listening on IPv4 address "127.30.8.20", port 59633
2026-08-11 15:09:26.258 UTC [15808] LOG: listening on Unix socket "/tmp/.s.PGSQL.59633"
2026-08-11 15:09:26.284 UTC [15815] LOG: database system was shut down at 2026-08-11 15:09:24 UTC
2026-08-11 15:09:26.289 UTC [15808] LOG: database system is ready to accept connections
I20260811 15:09:27.389555 30752 mini_postgres.cc:96] Postgres bound to 59633
2026-08-11 15:09:27.398 UTC [15824] FATAL: database "slave" does not exist
127.30.8.20:59633 - accepting connections
I20260811 15:09:27.399672 30752 mini_ranger.cc:161] Starting Ranger...
I20260811 15:09:27.417873 30752 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 15:09:27.908497 30752 mini_ranger.cc:87] Created ranger Postgres database
I20260811 15:09:27.908599 30752 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ranger-admin
2026-08-11 15:09:28,359 [I] DB FLAVOR :POSTGRES
2026-08-11 15:09:28,359 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 15:09:28,359 [I] Checking connection..
2026-08-11 15:09:28,360 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 15:09:28,678 [I] Checking connection passed.
2026-08-11 15:09:28,678 [I] --------- Verifying version history table ---------
2026-08-11 15:09:28,678 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 15:09:28,960 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 15:09:28,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 15:09:29,260 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 15:09:29,261 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 15:09:29,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 15:09:29.524 UTC [15933] WARNING: there is no transaction in progress
2026-08-11 15:09:29,536 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 15:09:29,536 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 15:09:29,821 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 15:09:29,821 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 15:09:29,821 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 15:09:30,114 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 15:09:30,381 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1vmg.c.gcp-upstream.internal','N') ;"
2026-08-11 15:09:30,621 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 15:09:30,874 [I] Table x_portal_user does not exist in database ranger
2026-08-11 15:09:30,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 15:09:31,149 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 15:09:31,149 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 15:09:31,423 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 15:09:31,691 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 15:09:31,691 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 15:09:32.608 UTC [16102] WARNING: there is no transaction in progress
2026-08-11 15:09:32.626 UTC [16102] WARNING: there is no transaction in progress
2026-08-11 15:09:32.644 UTC [16102] WARNING: there is no transaction in progress
2026-08-11 15:09:32.662 UTC [16102] WARNING: there is no transaction in progress
2026-08-11 15:09:32.682 UTC [16102] WARNING: there is no transaction in progress
2026-08-11 15:09:32.702 UTC [16102] WARNING: there is no transaction in progress
2026-08-11 15:09:32.771 UTC [16102] WARNING: there is no transaction in progress
2026-08-11 15:09:32.778 UTC [16102] WARNING: there is no transaction in progress
2026-08-11 15:09:32.795 UTC [16102] WARNING: there is no transaction in progress
2026-08-11 15:09:32.807 UTC [16102] WARNING: there is no transaction in progress
2026-08-11 15:09:32.819 UTC [16102] WARNING: there is no transaction in progress
2026-08-11 15:09:32.835 UTC [16102] WARNING: there is no transaction in progress
2026-08-11 15:09:32.846 UTC [16102] WARNING: there is no transaction in progress
2026-08-11 15:09:32.852 UTC [16102] WARNING: there is no transaction in progress
2026-08-11 15:09:32.859 UTC [16102] WARNING: there is no transaction in progress
2026-08-11 15:09:32.866 UTC [16102] WARNING: there is no transaction in progress
2026-08-11 15:09:33,558 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 15:09:33,558 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 15:09:33,817 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 15:09:33,817 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 15:09:34,095 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 15:09:34,095 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 15:09:34,372 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 15:09:34,372 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 15:09:34,647 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 15:09:34,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1vmg.c.gcp-upstream.internal';"
2026-08-11 15:09:35,180 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 15:09:35,181 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 15:09:35,464 [I] DB_PATCHES have already been applied
I20260811 15:09:35.470733 30752 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ranger-admin:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskxgyPSN/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 15:09:35.470813 30752 mini_ranger.cc:193] Using host: 127.30.8.20
I20260811 15:09:35.475749 30752 mini_ranger.cc:227] Ranger admin URL: http://127.30.8.20:47243
Aug 11, 2026 3:09:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 3:09:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 3:09:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 3:09:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 3:09:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 3:09:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 3:09:36 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 15:09:36.741509 30752 mini_ranger.cc:160] Time spent starting Ranger: real 9.342s user 0.005s sys 0.010s
Aug 11, 2026 3:09:57 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 3:09:57 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 11, 2026 3:09:57 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 3:09:57 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 3:09:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 3:09:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 3:09:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 3:09:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 3:09:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 3:09:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 3:09:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 3:09:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 3:09:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 3:09:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 3:09:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 3:09:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 3:09:57 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 3:09:58 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 15:09:58.698087 30752 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.8.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.8.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.8.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/rangerkms_127.30.8.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.8.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/rangerkms_127.30.8.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.8.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.8.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.8.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/HTTP_127.30.8.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.8.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/HTTP_127.30.8.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 15:09:58 dist-test-slave-1vmg krb5kdc[15762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460998, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 15:09:58.809813 30752 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 15:09:58.821853 30752 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 15:09:59.593578 30752 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 15:09:59.593662 30752 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ranger-kms
I20260811 15:09:59.593744 30752 mini_ranger_kms.cc:227] Using postgres at 127.30.8.20:59633
2026-08-11 15:10:00,126 [I] DB FLAVOR :POSTGRES
2026-08-11 15:10:00,126 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 15:10:00,126 [I] Checking connection
2026-08-11 15:10:00,126 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 15:10:00,385 [I] connection success
2026-08-11 15:10:00,385 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 15:10:00,385 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 15:10:00,385 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 15:10:00,634 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 15:10:00,635 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 15:10:00,635 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 15:10:00,635 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:59633/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 15:10:00,927 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 15:10:01.391873 30752 mini_ranger_kms.cc:310] Created kms service
I20260811 15:10:01.605065 30752 mini_ranger_kms.cc:326] Created kudu user
I20260811 15:10:01.645812 30752 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 15:10:01 dist-test-slave-1vmg krb5kdc[15762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786460998, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
I20260811 15:10:02.051427 30752 mini_ranger_kms.cc:384] Added ranger policy
I20260811 15:10:02.051613 30752 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ranger-kms:/tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 15:10:02.051641 30752 mini_ranger_kms.cc:240] Using host: 127.30.8.20
I20260811 15:10:02.056470 30752 mini_ranger_kms.cc:276] Ranger KMS PID: 16376
I20260811 15:10:02.056622 30752 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.8.20:36555
Aug 11, 2026 3:10:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 3:10:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 3:10:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 3:10:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 3:10:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 3:10:03 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 15:10:04 dist-test-slave-1vmg krb5kdc[15762](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786461004, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.8.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:10:04.565876 30752 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.756s user 0.012s sys 0.254s
I20260811 15:10:04.566005 30752 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 15:10:04.566136 30752 mini_ranger_kms.cc:401] 127.30.8.20:36555/kms/v1/keys
Aug 11 15:10:04 dist-test-slave-1vmg krb5kdc[15762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461004, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.8.20@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
Aug 11 15:10:05 dist-test-slave-1vmg krb5kdc[15762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461004, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.8.20@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
Aug 11 15:10:07 dist-test-slave-1vmg krb5kdc[15762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:10:07.995828 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:42533
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:36113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:42533
--ranger_config_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.8.20:36555/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:10:08.125288 16457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:10:08.125610 16457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:10:08.125653 16457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:10:08.129921 16457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:10:08.130012 16457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:10:08.130033 16457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:10:08.130048 16457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:10:08.130069 16457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:10:08.136368 16457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.8.20:36555/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:42533
--ranger_config_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:42533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:10:08.137676 16457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:10:08.138850 16457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:10:08.148653 16464 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:10:08.148931 16457 server_base.cc:1034] running on GCE node
W20260811 15:10:08.149111 16466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:10:08.148680 16463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:10:08.150517 16457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:10:08.151921 16457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:10:08.153117 16457 hybrid_clock.cc:648] HybridClock initialized: now 1786461008153095 us; error 46 us; skew 500 ppm
Aug 11 15:10:08 dist-test-slave-1vmg krb5kdc[15762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:10:08.156610 16457 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:10:08.157970 16457 webserver.cc:460] Webserver started at http://127.30.8.62:40155/ using document root <none> and password file <none>
I20260811 15:10:08.158739 16457 fs_manager.cc:359] Metadata directory not provided
I20260811 15:10:08.158830 16457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:10:08.159154 16457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 15:10:08 dist-test-slave-1vmg krb5kdc[15762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461008, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
I20260811 15:10:08.220906 16457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "473c1cba35dc489fb37cf9051b172362"
format_stamp: "Formatted at 2026-08-11 15:10:08 on dist-test-slave-1vmg"
server_key: "837e792bf16395a440e964a732f69f26"
server_key_iv: "0dad4aff16721ac03c6fe31ebbdb8ff5"
server_key_version: "kuduclusterkey@0"
I20260811 15:10:08.222121 16457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "473c1cba35dc489fb37cf9051b172362"
format_stamp: "Formatted at 2026-08-11 15:10:08 on dist-test-slave-1vmg"
server_key: "837e792bf16395a440e964a732f69f26"
server_key_iv: "0dad4aff16721ac03c6fe31ebbdb8ff5"
server_key_version: "kuduclusterkey@0"
I20260811 15:10:08.229519 16457 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.009s sys 0.000s
I20260811 15:10:08.250406 16477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:10:08.252194 16457 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:10:08.252377 16457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "473c1cba35dc489fb37cf9051b172362"
format_stamp: "Formatted at 2026-08-11 15:10:08 on dist-test-slave-1vmg"
server_key: "837e792bf16395a440e964a732f69f26"
server_key_iv: "0dad4aff16721ac03c6fe31ebbdb8ff5"
server_key_version: "kuduclusterkey@0"
I20260811 15:10:08.252544 16457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:10:08.291738 16457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:10:08.295962 16457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:10:08.296227 16457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:10:08.307317 16457 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:42533
I20260811 15:10:08.307549 16561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:42533 every 8 connection(s)
I20260811 15:10:08.308763 16457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:10:08.312489 16562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:10:08.318658 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 16457
I20260811 15:10:08.318826 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:10:08.319159 30752 external_mini_cluster.cc:1428] Setting key a9545301db49bf8e6ac34e8d18dcb50c
I20260811 15:10:08.323793 16562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362: Bootstrap starting.
I20260811 15:10:08.327111 16562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362: Neither blocks nor log segments found. Creating new log.
I20260811 15:10:08.328141 16562 log.cc:824] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362: Log is configured to *not* fsync() on all Append() calls
I20260811 15:10:08.333124 16562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362: No bootstrap required, opened a new log
I20260811 15:10:08.338975 16562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473c1cba35dc489fb37cf9051b172362" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42533 } }
I20260811 15:10:08.339476 16562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:10:08.339614 16562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 473c1cba35dc489fb37cf9051b172362, State: Initialized, Role: FOLLOWER
Aug 11 15:10:08 dist-test-slave-1vmg krb5kdc[15762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461007, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:10:08.340428 16562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473c1cba35dc489fb37cf9051b172362" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42533 } }
I20260811 15:10:08.340644 16562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:10:08.340729 16562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:10:08.340859 16562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:10:08.342720 16562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473c1cba35dc489fb37cf9051b172362" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42533 } }
I20260811 15:10:08.343564 16562 leader_election.cc:302] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 473c1cba35dc489fb37cf9051b172362; no voters:
I20260811 15:10:08.344027 16562 leader_election.cc:288] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:10:08.344836 16570 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:10:08.345129 16570 raft_consensus.cc:686] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362 [term 1 LEADER]: Becoming Leader. State: Replica: 473c1cba35dc489fb37cf9051b172362, State: Running, Role: LEADER
I20260811 15:10:08.345554 16570 consensus_queue.cc:237] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473c1cba35dc489fb37cf9051b172362" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42533 } }
I20260811 15:10:08.345852 16562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:10:08.347059 16570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "473c1cba35dc489fb37cf9051b172362" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473c1cba35dc489fb37cf9051b172362" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42533 } } }
I20260811 15:10:08.347307 16570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362 [sys.catalog]: This master's current role is: LEADER
I20260811 15:10:08.347573 16570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 473c1cba35dc489fb37cf9051b172362. Latest consensus state: current_term: 1 leader_uuid: "473c1cba35dc489fb37cf9051b172362" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473c1cba35dc489fb37cf9051b172362" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42533 } } }
I20260811 15:10:08.347721 16570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362 [sys.catalog]: This master's current role is: LEADER
I20260811 15:10:08.348513 16562 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 15:10:08.348966 16567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:08.321147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57100 (local address 127.30.8.62:42533)
0811 15:10:08.321705 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0811 15:10:08.321727 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 15:10:08.328846 (+ 7119us) server_negotiation.cc:407] Connection header received
0811 15:10:08.329906 (+ 1060us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:10:08.329938 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:10:08.330445 (+ 507us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:10:08.330870 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:10:08.332007 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:08.333940 (+ 1933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:08.335436 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:08.336584 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:08.336871 (+ 287us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:08.341258 (+ 4387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:10:08.341268 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:10:08.341296 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:10:08.341355 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:10:08.344422 (+ 3067us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:10:08.345143 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:10:08.345149 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:10:08.345156 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:10:08.345314 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:10:08.345656 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:10:08.345660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:10:08.345662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:10:08.346293 (+ 631us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:10:08.346504 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 15:10:08.347525 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0811 15:10:08.347884 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":148,"threads_started":1}
I20260811 15:10:08.355255 16583 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:10:08.359860 16583 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:10:08.368376 16583 catalog_manager.cc:1347] Generated new cluster ID: 03749a9b471543f5abe3c1c44e18be8f
I20260811 15:10:08.368456 16583 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:10:08.411299 16583 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:10:08.412994 16583 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:10:08.420717 16583 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362: Generated new TSK 0
I20260811 15:10:08.421939 16583 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
I20260811 15:10:09.223714 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.8.20:36555/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:42533
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:36113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:10:09.406273 16635 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:10:09.406850 16635 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:10:09.407047 16635 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:10:09.413975 16635 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:10:09.414108 16635 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:10:09.414215 16635 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:10:09.424661 16635 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.8.20:36555/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:42533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16635
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:10:09.426271 16635 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:10:09.427781 16635 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:10:09.441717 16646 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:10:09.441807 16651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:10:09.441931 16645 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:10:09.443035 16635 server_base.cc:1034] running on GCE node
I20260811 15:10:09.443570 16635 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:10:09.444491 16635 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:10:09.445816 16635 hybrid_clock.cc:648] HybridClock initialized: now 1786461009445784 us; error 56 us; skew 500 ppm
Aug 11 15:10:09 dist-test-slave-1vmg krb5kdc[15762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461009, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:10:09.450248 16635 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:10:09.451603 16635 webserver.cc:460] Webserver started at http://127.30.8.1:44805/ using document root <none> and password file <none>
I20260811 15:10:09.452446 16635 fs_manager.cc:359] Metadata directory not provided
I20260811 15:10:09.452626 16635 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:10:09.452997 16635 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 15:10:09 dist-test-slave-1vmg krb5kdc[15762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461009, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
I20260811 15:10:09.484978 16635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "ff68e0446d2e47968f5200a86b558285"
format_stamp: "Formatted at 2026-08-11 15:10:09 on dist-test-slave-1vmg"
server_key: "bbb52367e6409b9d0c472c60d31128c8"
server_key_iv: "5e667e01f3802fc1d3fa560cbd9d35a6"
server_key_version: "kuduclusterkey@0"
I20260811 15:10:09.485846 16635 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "ff68e0446d2e47968f5200a86b558285"
format_stamp: "Formatted at 2026-08-11 15:10:09 on dist-test-slave-1vmg"
server_key: "bbb52367e6409b9d0c472c60d31128c8"
server_key_iv: "5e667e01f3802fc1d3fa560cbd9d35a6"
server_key_version: "kuduclusterkey@0"
I20260811 15:10:09.491859 16635 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260811 15:10:09.513746 16665 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:10:09.515597 16635 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:10:09.515749 16635 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "ff68e0446d2e47968f5200a86b558285"
format_stamp: "Formatted at 2026-08-11 15:10:09 on dist-test-slave-1vmg"
server_key: "bbb52367e6409b9d0c472c60d31128c8"
server_key_iv: "5e667e01f3802fc1d3fa560cbd9d35a6"
server_key_version: "kuduclusterkey@0"
I20260811 15:10:09.515898 16635 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:10:09.542668 16635 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:10:09.545934 16635 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:10:09.546208 16635 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:10:09.547039 16635 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:10:09.548285 16635 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:10:09.548347 16635 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:10:09.548411 16635 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:10:09.548467 16635 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:10:09.561923 16635 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:35545
I20260811 15:10:09.561955 16850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:35545 every 8 connection(s)
I20260811 15:10:09.563211 16635 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:10:09.567217 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 16635
I20260811 15:10:09.567348 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:10:09.567632 30752 external_mini_cluster.cc:1428] Setting key 919f094dcc6ab1b7266d064af93b02e2
Aug 11 15:10:09 dist-test-slave-1vmg krb5kdc[15762](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786461009, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 15:10:09 dist-test-slave-1vmg krb5kdc[15762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461009, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.8.2@KRBTEST.COM; defaulting to no policy
I20260811 15:10:09.590227 16856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:09.565709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:37843 (local address 127.30.8.62:42533)
0811 15:10:09.571116 (+ 5407us) server_negotiation.cc:206] Beginning negotiation
0811 15:10:09.571121 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:10:09.573725 (+ 2604us) server_negotiation.cc:407] Connection header received
0811 15:10:09.573779 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:10:09.573782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:10:09.573838 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:10:09.573922 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:10:09.575736 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:09.576615 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:09.578461 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:09.578845 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:09.578963 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:09.584792 (+ 5829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:10:09.584797 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:10:09.584800 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:10:09.584836 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:10:09.586977 (+ 2141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:10:09.587782 (+ 805us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:10:09.587786 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:10:09.587788 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:10:09.587902 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:10:09.588525 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:10:09.588530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:10:09.588532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:10:09.588857 (+ 325us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:10:09.588960 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 15:10:09.589848 (+ 888us) server_negotiation.cc:299] Negotiation successful
0811 15:10:09.590012 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5311,"thread_start_us":119,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.2@KRBTEST.COM" created.
I20260811 15:10:09.592352 16857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:09.566145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:42533 (local address 127.30.8.1:37843)
0811 15:10:09.572109 (+ 5964us) negotiation.cc:107] Waiting for socket to connect
0811 15:10:09.572187 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 15:10:09.573370 (+ 1183us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:10:09.574261 (+ 891us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:10:09.574272 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:10:09.574690 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:10:09.575532 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:10:09.575554 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:10:09.576804 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:10:09.576807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:10:09.578314 (+ 1507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:10:09.578327 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:10:09.579202 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:10:09.579207 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:10:09.579432 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:09.582376 (+ 2944us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:10:09.582406 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:10:09.584514 (+ 2108us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:10:09.587166 (+ 2652us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:10:09.587176 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:10:09.587198 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:10:09.587632 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:10:09.588044 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:10:09.588048 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:10:09.588050 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:10:09.588270 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:10:09.589044 (+ 774us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:10:09.589053 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:10:09.589478 (+ 425us) client_negotiation.cc:769] Sending connection context
0811 15:10:09.590739 (+ 1261us) client_negotiation.cc:240] Negotiation successful
0811 15:10:09.591170 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5696,"thread_start_us":151,"threads_started":1}
I20260811 15:10:09.594004 16853 heartbeater.cc:347] Connected to a master server at 127.30.8.62:42533
I20260811 15:10:09.594274 16853 heartbeater.cc:464] Registering TS with master...
I20260811 15:10:09.594900 16853 heartbeater.cc:511] Master 127.30.8.62:42533 requested a full tablet report, sending...
I20260811 15:10:09.596920 16502 ts_manager.cc:194] Registered new tserver with Master: ff68e0446d2e47968f5200a86b558285 (127.30.8.1:35545)
I20260811 15:10:09.598515 16502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:37843
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
I20260811 15:10:09.620698 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.8.20:36555/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:42533
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:36113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:10:09.782045 16862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:10:09.782508 16862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:10:09.782636 16862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:10:09.788810 16862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:10:09.788978 16862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:10:09.789126 16862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:10:09.797379 16862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.8.20:36555/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:42533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:10:09.799290 16862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:10:09.800786 16862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:10:09.814440 16875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:10:09.815100 16873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:10:09.815358 16870 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:10:09.817006 16862 server_base.cc:1034] running on GCE node
I20260811 15:10:09.817576 16862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:10:09.818511 16862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:10:09.819706 16862 hybrid_clock.cc:648] HybridClock initialized: now 1786461009819669 us; error 51 us; skew 500 ppm
Aug 11 15:10:09 dist-test-slave-1vmg krb5kdc[15762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461009, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:10:09.823827 16862 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:10:09.825057 16862 webserver.cc:460] Webserver started at http://127.30.8.2:32839/ using document root <none> and password file <none>
I20260811 15:10:09.825666 16862 fs_manager.cc:359] Metadata directory not provided
I20260811 15:10:09.825742 16862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:10:09.825968 16862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 15:10:09 dist-test-slave-1vmg krb5kdc[15762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461009, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
I20260811 15:10:09.842777 16862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "2af2d5f45d9f4add9422d8b9023343a5"
format_stamp: "Formatted at 2026-08-11 15:10:09 on dist-test-slave-1vmg"
server_key: "4f65b5350825b6a1b26e397278ed727d"
server_key_iv: "5ef5d6ffae2018c67019712105341491"
server_key_version: "kuduclusterkey@0"
I20260811 15:10:09.843667 16862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "2af2d5f45d9f4add9422d8b9023343a5"
format_stamp: "Formatted at 2026-08-11 15:10:09 on dist-test-slave-1vmg"
server_key: "4f65b5350825b6a1b26e397278ed727d"
server_key_iv: "5ef5d6ffae2018c67019712105341491"
server_key_version: "kuduclusterkey@0"
I20260811 15:10:09.849198 16862 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 15:10:09.882000 16887 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:10:09.883759 16862 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:10:09.883934 16862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "2af2d5f45d9f4add9422d8b9023343a5"
format_stamp: "Formatted at 2026-08-11 15:10:09 on dist-test-slave-1vmg"
server_key: "4f65b5350825b6a1b26e397278ed727d"
server_key_iv: "5ef5d6ffae2018c67019712105341491"
server_key_version: "kuduclusterkey@0"
I20260811 15:10:09.884104 16862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:10:09.900578 16862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:10:09.904578 16862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:10:09.904877 16862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:10:09.905779 16862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:10:09.907099 16862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:10:09.907200 16862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:10:09.907286 16862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:10:09.907334 16862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:10:09.924705 16862 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:36929
I20260811 15:10:09.926162 16862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:10:09.931655 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 16862
I20260811 15:10:09.931794 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:10:09.932102 30752 external_mini_cluster.cc:1428] Setting key 654f9f1f220f9c8b9844135852c75857
I20260811 15:10:09.933851 17058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:36929 every 8 connection(s)
Aug 11 15:10:09 dist-test-slave-1vmg krb5kdc[15762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461009, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:10:09.965953 16856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:09.933876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:47393 (local address 127.30.8.62:42533)
0811 15:10:09.934014 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 15:10:09.934019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:10:09.947312 (+ 13293us) server_negotiation.cc:407] Connection header received
0811 15:10:09.948987 (+ 1675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:10:09.948991 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:10:09.949065 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:10:09.949165 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:10:09.950939 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:09.951794 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:09.954977 (+ 3183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:09.955347 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:09.955434 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:09.959005 (+ 3571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:10:09.959009 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:10:09.959012 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:10:09.959057 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:10:09.961053 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:10:09.962016 (+ 963us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:10:09.962020 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:10:09.962022 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:10:09.962133 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:10:09.964444 (+ 2311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:10:09.964448 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:10:09.964450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:10:09.964709 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:10:09.964786 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 15:10:09.965626 (+ 840us) server_negotiation.cc:299] Negotiation successful
0811 15:10:09.965774 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
WARNING: no policy specified for kudu/127.30.8.3@KRBTEST.COM; defaulting to no policy
I20260811 15:10:09.970768 17061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:09.932600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:42533 (local address 127.30.8.2:47393)
0811 15:10:09.947045 (+ 14445us) negotiation.cc:107] Waiting for socket to connect
0811 15:10:09.947151 (+ 106us) client_negotiation.cc:174] Beginning negotiation
0811 15:10:09.948689 (+ 1538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:10:09.949538 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:10:09.949549 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:10:09.949947 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:10:09.950719 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:10:09.950741 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:10:09.953048 (+ 2307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:10:09.953053 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:10:09.954782 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:10:09.954795 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:10:09.955688 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:10:09.955692 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:10:09.955924 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:09.956694 (+ 770us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:10:09.956717 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:10:09.958737 (+ 2020us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:10:09.961423 (+ 2686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:10:09.961430 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:10:09.961451 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:10:09.961827 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:10:09.962262 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:10:09.962265 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:10:09.962268 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:10:09.964316 (+ 2048us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:10:09.964874 (+ 558us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:10:09.964882 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:10:09.965316 (+ 434us) client_negotiation.cc:769] Sending connection context
0811 15:10:09.966489 (+ 1173us) client_negotiation.cc:240] Negotiation successful
0811 15:10:09.967116 (+ 627us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13973,"thread_start_us":169,"threads_started":1}
I20260811 15:10:09.974555 17059 heartbeater.cc:347] Connected to a master server at 127.30.8.62:42533
I20260811 15:10:09.974843 17059 heartbeater.cc:464] Registering TS with master...
I20260811 15:10:09.975550 17059 heartbeater.cc:511] Master 127.30.8.62:42533 requested a full tablet report, sending...
I20260811 15:10:09.976908 16502 ts_manager.cc:194] Registered new tserver with Master: 2af2d5f45d9f4add9422d8b9023343a5 (127.30.8.2:36929)
I20260811 15:10:09.977710 16502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:47393
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
I20260811 15:10:10.020529 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.8.20:36555/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:42533
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:36113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 15:10:10 dist-test-slave-1vmg krb5kdc[15762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461009, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
W20260811 15:10:10.194895 17066 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:10:10.195779 17066 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:10:10.195897 17066 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:10:10.202062 17066 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:10:10.202209 17066 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:10:10.202344 17066 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:10:10.210376 17066 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.8.20:36555/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:42533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17066
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:10:10.211982 17066 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:10:10.213375 17066 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:10:10.229918 17073 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:10:10.230652 17072 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:10:10.231349 17075 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:10:10.237746 17066 server_base.cc:1034] running on GCE node
I20260811 15:10:10.238458 17066 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:10:10.239465 17066 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:10:10.246891 17066 hybrid_clock.cc:648] HybridClock initialized: now 1786461010246793 us; error 111 us; skew 500 ppm
Aug 11 15:10:10 dist-test-slave-1vmg krb5kdc[15762](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461010, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:10:10.251076 17066 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:10:10.252326 17066 webserver.cc:460] Webserver started at http://127.30.8.3:46247/ using document root <none> and password file <none>
I20260811 15:10:10.253105 17066 fs_manager.cc:359] Metadata directory not provided
I20260811 15:10:10.253249 17066 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:10:10.253535 17066 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 15:10:10 dist-test-slave-1vmg krb5kdc[15762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461010, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
I20260811 15:10:10.282940 17066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "6d5dc2aec08f4284ac0766eaed78c268"
format_stamp: "Formatted at 2026-08-11 15:10:10 on dist-test-slave-1vmg"
server_key: "266d051a44549d477195e87165ff9bd7"
server_key_iv: "65775de14046f3ac82a2cf378513f993"
server_key_version: "kuduclusterkey@0"
I20260811 15:10:10.283843 17066 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "6d5dc2aec08f4284ac0766eaed78c268"
format_stamp: "Formatted at 2026-08-11 15:10:10 on dist-test-slave-1vmg"
server_key: "266d051a44549d477195e87165ff9bd7"
server_key_iv: "65775de14046f3ac82a2cf378513f993"
server_key_version: "kuduclusterkey@0"
I20260811 15:10:10.301745 17066 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.004s sys 0.000s
I20260811 15:10:10.319643 17084 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:10:10.321714 17066 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:10:10.321931 17066 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "6d5dc2aec08f4284ac0766eaed78c268"
format_stamp: "Formatted at 2026-08-11 15:10:10 on dist-test-slave-1vmg"
server_key: "266d051a44549d477195e87165ff9bd7"
server_key_iv: "65775de14046f3ac82a2cf378513f993"
server_key_version: "kuduclusterkey@0"
I20260811 15:10:10.322180 17066 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:10:10.340176 17066 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:10:10.344151 17066 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:10:10.344424 17066 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:10:10.345189 17066 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:10:10.346336 17066 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:10:10.346442 17066 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:10:10.346539 17066 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:10:10.346604 17066 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:10:10.361392 17066 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:41595
I20260811 15:10:10.362957 17066 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:10:10.364969 17261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:41595 every 8 connection(s)
I20260811 15:10:10.372090 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 17066
I20260811 15:10:10.372226 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:10:10.372509 30752 external_mini_cluster.cc:1428] Setting key 0c472f306e7eb76d5bbfc25b4fd5b1fd
Aug 11 15:10:10 dist-test-slave-1vmg krb5kdc[15762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461010, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:10:10.389819 16856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:10.368660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:35871 (local address 127.30.8.62:42533)
0811 15:10:10.368786 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 15:10:10.368791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:10:10.369689 (+ 898us) server_negotiation.cc:407] Connection header received
0811 15:10:10.371007 (+ 1318us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:10:10.371011 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:10:10.371080 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:10:10.371133 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:10:10.376583 (+ 5450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.377416 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.379149 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.379459 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.379533 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:10.382805 (+ 3272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:10:10.382810 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:10:10.382813 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:10:10.382844 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:10:10.384702 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:10:10.387807 (+ 3105us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:10:10.387810 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:10:10.387812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:10:10.387909 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:10:10.388339 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:10:10.388342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:10:10.388344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:10:10.388600 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:10:10.388680 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 15:10:10.389484 (+ 804us) server_negotiation.cc:299] Negotiation successful
0811 15:10:10.389637 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 15:10:10.391829 17265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:10.367463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:42533 (local address 127.30.8.3:35871)
0811 15:10:10.369517 (+ 2054us) negotiation.cc:107] Waiting for socket to connect
0811 15:10:10.369571 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 15:10:10.370688 (+ 1117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:10:10.371577 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:10:10.371586 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:10:10.375602 (+ 4016us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:10:10.376387 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:10:10.376414 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.377588 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.377591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:10:10.379000 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:10:10.379013 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.379783 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.379787 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:10:10.380007 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:10.380763 (+ 756us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:10:10.380787 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:10:10.382587 (+ 1800us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:10:10.387241 (+ 4654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:10:10.387250 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:10:10.387273 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:10:10.387687 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:10:10.388025 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:10:10.388027 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:10:10.388029 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:10:10.388232 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:10:10.388787 (+ 555us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:10:10.388795 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:10:10.389191 (+ 396us) client_negotiation.cc:769] Sending connection context
0811 15:10:10.390329 (+ 1138us) client_negotiation.cc:240] Negotiation successful
0811 15:10:10.390709 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1827,"thread_start_us":166,"threads_started":1}
I20260811 15:10:10.393389 17263 heartbeater.cc:347] Connected to a master server at 127.30.8.62:42533
I20260811 15:10:10.393646 17263 heartbeater.cc:464] Registering TS with master...
I20260811 15:10:10.394249 17263 heartbeater.cc:511] Master 127.30.8.62:42533 requested a full tablet report, sending...
I20260811 15:10:10.395493 16502 ts_manager.cc:194] Registered new tserver with Master: 6d5dc2aec08f4284ac0766eaed78c268 (127.30.8.3:41595)
I20260811 15:10:10.396235 16502 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:35871
I20260811 15:10:10.400149 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 15:10:10.412971 16856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:10.402964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57104 (local address 127.30.8.62:42533)
0811 15:10:10.403090 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 15:10:10.403094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:10:10.403622 (+ 528us) server_negotiation.cc:407] Connection header received
0811 15:10:10.403797 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:10:10.403800 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:10:10.403850 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:10:10.403898 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:10:10.404882 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.405684 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.406941 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.407233 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.407311 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:10.409206 (+ 1895us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:10:10.409210 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:10:10.409213 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:10:10.409243 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:10:10.410954 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:10:10.411468 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:10:10.411472 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:10:10.411474 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:10:10.411570 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:10:10.411888 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:10:10.411892 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:10:10.411893 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:10:10.412159 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:10:10.412267 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 15:10:10.412669 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 15:10:10.412801 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 15:10:10.417320 16502 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57104:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:10:10.422228 16502 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:10:10.452271 17288 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:10.436710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:41595 (local address 127.0.0.1:34236)
0811 15:10:10.442250 (+ 5540us) negotiation.cc:107] Waiting for socket to connect
0811 15:10:10.442314 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 15:10:10.442465 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:10:10.444781 (+ 2316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:10:10.444788 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:10:10.444809 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:10:10.444957 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:10:10.444963 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.446628 (+ 1665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.446632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:10:10.448974 (+ 2342us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:10:10.448987 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.450784 (+ 1797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.450788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:10:10.451774 (+ 986us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:10.451821 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 15:10:10.451944 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 15:10:10.452100 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5338,"thread_start_us":140,"threads_started":1}
I20260811 15:10:10.454743 17282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:10.437755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34236 (local address 127.30.8.3:41595)
0811 15:10:10.441949 (+ 4194us) server_negotiation.cc:206] Beginning negotiation
0811 15:10:10.441962 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 15:10:10.442621 (+ 659us) server_negotiation.cc:407] Connection header received
0811 15:10:10.442717 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:10:10.442722 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:10:10.442995 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:10:10.443166 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:10:10.445112 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.446480 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.449477 (+ 2997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.450428 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.450540 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:10.450714 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 15:10:10.454428 (+ 3714us) server_negotiation.cc:299] Negotiation successful
0811 15:10:10.454572 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4042,"thread_start_us":134,"threads_started":1}
I20260811 15:10:10.457710 17281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:10.439726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:36929 (local address 127.0.0.1:41518)
0811 15:10:10.443432 (+ 3706us) negotiation.cc:107] Waiting for socket to connect
0811 15:10:10.443444 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 15:10:10.443531 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:10:10.449125 (+ 5594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:10:10.449128 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:10:10.449148 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:10:10.449234 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:10:10.449240 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.455055 (+ 5815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.455058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:10:10.456130 (+ 1072us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:10:10.456138 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.457294 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.457298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:10:10.457393 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:10.457421 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 15:10:10.457468 (+ 47us) client_negotiation.cc:240] Negotiation successful
0811 15:10:10.457544 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3615,"thread_start_us":93,"threads_started":1}
I20260811 15:10:10.458427 17285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:10.439941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41518 (local address 127.30.8.2:36929)
0811 15:10:10.448403 (+ 8462us) server_negotiation.cc:206] Beginning negotiation
0811 15:10:10.448424 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 15:10:10.448456 (+ 32us) server_negotiation.cc:407] Connection header received
0811 15:10:10.448545 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:10:10.448551 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:10:10.448843 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:10:10.449034 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:10:10.452663 (+ 3629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.453925 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.456307 (+ 2382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.457148 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.457962 (+ 814us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:10.458134 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 15:10:10.458191 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 15:10:10.458287 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8326,"thread_start_us":82,"threads_started":1}
I20260811 15:10:10.460057 17284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:10.440470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:35545 (local address 127.0.0.1:52980)
0811 15:10:10.444534 (+ 4064us) negotiation.cc:107] Waiting for socket to connect
0811 15:10:10.444548 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:10:10.444637 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:10:10.455740 (+ 11103us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:10:10.455744 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:10:10.455764 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:10:10.455857 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:10:10.455863 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.457595 (+ 1732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.457598 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:10:10.458658 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:10:10.458669 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.459643 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.459647 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:10:10.459763 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:10.459792 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 15:10:10.459846 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 15:10:10.459920 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3999,"thread_start_us":108,"threads_started":1}
I20260811 15:10:10.461119 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:10.440639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52980 (local address 127.30.8.1:35545)
0811 15:10:10.455166 (+ 14527us) server_negotiation.cc:206] Beginning negotiation
0811 15:10:10.455174 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 15:10:10.455215 (+ 41us) server_negotiation.cc:407] Connection header received
0811 15:10:10.455305 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:10:10.455312 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:10:10.455567 (+ 255us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:10:10.456004 (+ 437us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:10:10.456146 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.457477 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.458787 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.459501 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.460637 (+ 1136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:10.460812 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 15:10:10.460871 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 15:10:10.460978 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14391,"thread_start_us":104,"threads_started":1}
I20260811 15:10:10.462839 16968 tablet_service.cc:1467] Processing CreateTablet for tablet 2addf7d5c3584027be0340e7ddaec144 (DEFAULT_TABLE table=test-table [id=90d8eddb64fe4f50843811ae2c6c7887]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:10:10.464022 17163 tablet_service.cc:1467] Processing CreateTablet for tablet 2addf7d5c3584027be0340e7ddaec144 (DEFAULT_TABLE table=test-table [id=90d8eddb64fe4f50843811ae2c6c7887]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:10:10.464090 16722 tablet_service.cc:1467] Processing CreateTablet for tablet 2addf7d5c3584027be0340e7ddaec144 (DEFAULT_TABLE table=test-table [id=90d8eddb64fe4f50843811ae2c6c7887]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:10:10.465322 16722 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2addf7d5c3584027be0340e7ddaec144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:10:10.465334 16968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2addf7d5c3584027be0340e7ddaec144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:10:10.467813 17163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2addf7d5c3584027be0340e7ddaec144. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:10:10.489213 17297 tablet_bootstrap.cc:492] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268: Bootstrap starting.
I20260811 15:10:10.489917 17299 tablet_bootstrap.cc:492] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5: Bootstrap starting.
I20260811 15:10:10.493197 17297 tablet_bootstrap.cc:654] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268: Neither blocks nor log segments found. Creating new log.
I20260811 15:10:10.496922 17296 tablet_bootstrap.cc:492] T 2addf7d5c3584027be0340e7ddaec144 P ff68e0446d2e47968f5200a86b558285: Bootstrap starting.
I20260811 15:10:10.497607 17299 tablet_bootstrap.cc:654] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5: Neither blocks nor log segments found. Creating new log.
I20260811 15:10:10.499327 17299 log.cc:824] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5: Log is configured to *not* fsync() on all Append() calls
I20260811 15:10:10.500466 17296 tablet_bootstrap.cc:654] T 2addf7d5c3584027be0340e7ddaec144 P ff68e0446d2e47968f5200a86b558285: Neither blocks nor log segments found. Creating new log.
I20260811 15:10:10.501639 17296 log.cc:824] T 2addf7d5c3584027be0340e7ddaec144 P ff68e0446d2e47968f5200a86b558285: Log is configured to *not* fsync() on all Append() calls
I20260811 15:10:10.503701 17297 log.cc:824] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268: Log is configured to *not* fsync() on all Append() calls
I20260811 15:10:10.507195 17296 tablet_bootstrap.cc:492] T 2addf7d5c3584027be0340e7ddaec144 P ff68e0446d2e47968f5200a86b558285: No bootstrap required, opened a new log
I20260811 15:10:10.509653 17299 tablet_bootstrap.cc:492] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5: No bootstrap required, opened a new log
I20260811 15:10:10.509995 17296 ts_tablet_manager.cc:1397] T 2addf7d5c3584027be0340e7ddaec144 P ff68e0446d2e47968f5200a86b558285: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.005s
I20260811 15:10:10.510303 17299 ts_tablet_manager.cc:1397] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.000s
Aug 11 15:10:10 dist-test-slave-1vmg krb5kdc[15762](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461009, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
I20260811 15:10:10.518409 17296 raft_consensus.cc:348] T 2addf7d5c3584027be0340e7ddaec144 P ff68e0446d2e47968f5200a86b558285 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d5dc2aec08f4284ac0766eaed78c268" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41595 } } peers { permanent_uuid: "2af2d5f45d9f4add9422d8b9023343a5" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 36929 } } peers { permanent_uuid: "ff68e0446d2e47968f5200a86b558285" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35545 } }
I20260811 15:10:10.521173 17297 tablet_bootstrap.cc:492] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268: No bootstrap required, opened a new log
I20260811 15:10:10.521477 17297 ts_tablet_manager.cc:1397] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268: Time spent bootstrapping tablet: real 0.033s user 0.007s sys 0.000s
I20260811 15:10:10.521638 17299 raft_consensus.cc:348] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d5dc2aec08f4284ac0766eaed78c268" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41595 } } peers { permanent_uuid: "2af2d5f45d9f4add9422d8b9023343a5" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 36929 } } peers { permanent_uuid: "ff68e0446d2e47968f5200a86b558285" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35545 } }
I20260811 15:10:10.522119 17299 raft_consensus.cc:374] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:10:10.522179 17299 raft_consensus.cc:729] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2af2d5f45d9f4add9422d8b9023343a5, State: Initialized, Role: FOLLOWER
I20260811 15:10:10.522482 17296 raft_consensus.cc:374] T 2addf7d5c3584027be0340e7ddaec144 P ff68e0446d2e47968f5200a86b558285 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:10:10.522547 17296 raft_consensus.cc:729] T 2addf7d5c3584027be0340e7ddaec144 P ff68e0446d2e47968f5200a86b558285 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff68e0446d2e47968f5200a86b558285, State: Initialized, Role: FOLLOWER
I20260811 15:10:10.523468 17299 consensus_queue.cc:260] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d5dc2aec08f4284ac0766eaed78c268" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41595 } } peers { permanent_uuid: "2af2d5f45d9f4add9422d8b9023343a5" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 36929 } } peers { permanent_uuid: "ff68e0446d2e47968f5200a86b558285" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35545 } }
I20260811 15:10:10.523716 17296 consensus_queue.cc:260] T 2addf7d5c3584027be0340e7ddaec144 P ff68e0446d2e47968f5200a86b558285 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d5dc2aec08f4284ac0766eaed78c268" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41595 } } peers { permanent_uuid: "2af2d5f45d9f4add9422d8b9023343a5" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 36929 } } peers { permanent_uuid: "ff68e0446d2e47968f5200a86b558285" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35545 } }
I20260811 15:10:10.526244 17296 ts_tablet_manager.cc:1428] T 2addf7d5c3584027be0340e7ddaec144 P ff68e0446d2e47968f5200a86b558285: Time spent starting tablet: real 0.013s user 0.006s sys 0.000s
I20260811 15:10:10.526512 17297 raft_consensus.cc:348] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d5dc2aec08f4284ac0766eaed78c268" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41595 } } peers { permanent_uuid: "2af2d5f45d9f4add9422d8b9023343a5" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 36929 } } peers { permanent_uuid: "ff68e0446d2e47968f5200a86b558285" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35545 } }
I20260811 15:10:10.526887 17297 raft_consensus.cc:374] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:10:10.527022 17297 raft_consensus.cc:729] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d5dc2aec08f4284ac0766eaed78c268, State: Initialized, Role: FOLLOWER
I20260811 15:10:10.527289 17299 ts_tablet_manager.cc:1428] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5: Time spent starting tablet: real 0.017s user 0.007s sys 0.000s
I20260811 15:10:10.527580 17297 consensus_queue.cc:260] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d5dc2aec08f4284ac0766eaed78c268" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41595 } } peers { permanent_uuid: "2af2d5f45d9f4add9422d8b9023343a5" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 36929 } } peers { permanent_uuid: "ff68e0446d2e47968f5200a86b558285" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35545 } }
I20260811 15:10:10.527966 17059 heartbeater.cc:503] Master 127.30.8.62:42533 was elected leader, sending a full tablet report...
I20260811 15:10:10.528646 17297 ts_tablet_manager.cc:1428] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 15:10:10.531566 17263 heartbeater.cc:503] Master 127.30.8.62:42533 was elected leader, sending a full tablet report...
I20260811 15:10:10.534314 16853 heartbeater.cc:503] Master 127.30.8.62:42533 was elected leader, sending a full tablet report...
W20260811 15:10:10.564975 16854 tablet.cc:2369] T 2addf7d5c3584027be0340e7ddaec144 P ff68e0446d2e47968f5200a86b558285: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:10:10.584708 17308 raft_consensus.cc:482] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:10:10.585035 17308 raft_consensus.cc:504] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d5dc2aec08f4284ac0766eaed78c268" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41595 } } peers { permanent_uuid: "2af2d5f45d9f4add9422d8b9023343a5" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 36929 } } peers { permanent_uuid: "ff68e0446d2e47968f5200a86b558285" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35545 } }
I20260811 15:10:10.586669 17308 leader_election.cc:288] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2af2d5f45d9f4add9422d8b9023343a5 (127.30.8.2:36929), ff68e0446d2e47968f5200a86b558285 (127.30.8.1:35545)
I20260811 15:10:10.591449 17265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:10.586949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:36929 (local address 127.30.8.3:59933)
0811 15:10:10.587270 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 15:10:10.587286 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:10:10.587390 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:10:10.587678 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:10:10.587681 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:10:10.587720 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:10:10.587852 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:10:10.587859 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.588901 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.588905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:10:10.590076 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:10:10.590089 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.590994 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.590998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:10:10.591098 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:10.591125 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 15:10:10.591210 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 15:10:10.591296 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230}
I20260811 15:10:10.591480 17285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:10.587261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:59933 (local address 127.30.8.2:36929)
0811 15:10:10.587406 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 15:10:10.587411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:10:10.587452 (+ 41us) server_negotiation.cc:407] Connection header received
0811 15:10:10.587517 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:10:10.587520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:10:10.587576 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:10:10.587649 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:10:10.587991 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.588778 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.590251 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.590836 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.590979 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:10.591054 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 15:10:10.591253 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 15:10:10.591346 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 15:10:10.591809 17289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:10.587472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:56271 (local address 127.30.8.1:35545)
0811 15:10:10.587847 (+ 375us) server_negotiation.cc:206] Beginning negotiation
0811 15:10:10.587851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:10:10.587861 (+ 10us) server_negotiation.cc:407] Connection header received
0811 15:10:10.587903 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:10:10.587905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:10:10.587948 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:10:10.588018 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:10:10.588385 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.589067 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.590519 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.591132 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.591204 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:10.591266 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 15:10:10.591631 (+ 365us) server_negotiation.cc:299] Negotiation successful
0811 15:10:10.591718 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293}
I20260811 15:10:10.592137 17312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:10.586981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:35545 (local address 127.30.8.3:56271)
0811 15:10:10.587553 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 15:10:10.587567 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:10:10.587652 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:10:10.588122 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:10:10.588125 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:10:10.588141 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:10:10.588254 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:10:10.588261 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.589209 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.589212 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:10:10.590383 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:10:10.590394 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.591365 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.591369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:10:10.591480 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:10.591505 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 15:10:10.591926 (+ 421us) client_negotiation.cc:240] Negotiation successful
0811 15:10:10.591994 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":509,"mutex_wait_us":17,"thread_start_us":132,"threads_started":1}
I20260811 15:10:10.592754 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2addf7d5c3584027be0340e7ddaec144" candidate_uuid: "6d5dc2aec08f4284ac0766eaed78c268" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff68e0446d2e47968f5200a86b558285" is_pre_election: true
I20260811 15:10:10.592775 16996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2addf7d5c3584027be0340e7ddaec144" candidate_uuid: "6d5dc2aec08f4284ac0766eaed78c268" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2af2d5f45d9f4add9422d8b9023343a5" is_pre_election: true
I20260811 15:10:10.593098 16783 raft_consensus.cc:2454] T 2addf7d5c3584027be0340e7ddaec144 P ff68e0446d2e47968f5200a86b558285 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d5dc2aec08f4284ac0766eaed78c268 in term 0.
I20260811 15:10:10.593099 16996 raft_consensus.cc:2454] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6d5dc2aec08f4284ac0766eaed78c268 in term 0.
I20260811 15:10:10.593605 17091 leader_election.cc:302] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5dc2aec08f4284ac0766eaed78c268, ff68e0446d2e47968f5200a86b558285; no voters:
I20260811 15:10:10.594113 17308 raft_consensus.cc:2781] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:10:10.594209 17308 raft_consensus.cc:482] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:10:10.594247 17308 raft_consensus.cc:3037] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:10:10.595695 17308 raft_consensus.cc:504] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d5dc2aec08f4284ac0766eaed78c268" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41595 } } peers { permanent_uuid: "2af2d5f45d9f4add9422d8b9023343a5" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 36929 } } peers { permanent_uuid: "ff68e0446d2e47968f5200a86b558285" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35545 } }
I20260811 15:10:10.596122 17308 leader_election.cc:288] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [CANDIDATE]: Term 1 election: Requested vote from peers 2af2d5f45d9f4add9422d8b9023343a5 (127.30.8.2:36929), ff68e0446d2e47968f5200a86b558285 (127.30.8.1:35545)
I20260811 15:10:10.596546 16783 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2addf7d5c3584027be0340e7ddaec144" candidate_uuid: "6d5dc2aec08f4284ac0766eaed78c268" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff68e0446d2e47968f5200a86b558285"
I20260811 15:10:10.596536 16996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2addf7d5c3584027be0340e7ddaec144" candidate_uuid: "6d5dc2aec08f4284ac0766eaed78c268" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2af2d5f45d9f4add9422d8b9023343a5"
I20260811 15:10:10.596673 16783 raft_consensus.cc:3037] T 2addf7d5c3584027be0340e7ddaec144 P ff68e0446d2e47968f5200a86b558285 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:10:10.596673 16996 raft_consensus.cc:3037] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:10:10.597922 16783 raft_consensus.cc:2454] T 2addf7d5c3584027be0340e7ddaec144 P ff68e0446d2e47968f5200a86b558285 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5dc2aec08f4284ac0766eaed78c268 in term 1.
I20260811 15:10:10.598340 16996 raft_consensus.cc:2454] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6d5dc2aec08f4284ac0766eaed78c268 in term 1.
I20260811 15:10:10.598392 17091 leader_election.cc:302] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6d5dc2aec08f4284ac0766eaed78c268, ff68e0446d2e47968f5200a86b558285; no voters:
I20260811 15:10:10.598559 17308 raft_consensus.cc:2781] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:10:10.598810 17308 raft_consensus.cc:686] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [term 1 LEADER]: Becoming Leader. State: Replica: 6d5dc2aec08f4284ac0766eaed78c268, State: Running, Role: LEADER
I20260811 15:10:10.599301 17308 consensus_queue.cc:237] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d5dc2aec08f4284ac0766eaed78c268" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41595 } } peers { permanent_uuid: "2af2d5f45d9f4add9422d8b9023343a5" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 36929 } } peers { permanent_uuid: "ff68e0446d2e47968f5200a86b558285" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35545 } }
I20260811 15:10:10.603303 16502 catalog_manager.cc:5495] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6d5dc2aec08f4284ac0766eaed78c268 (127.30.8.3). New cstate: current_term: 1 leader_uuid: "6d5dc2aec08f4284ac0766eaed78c268" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6d5dc2aec08f4284ac0766eaed78c268" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2af2d5f45d9f4add9422d8b9023343a5" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 36929 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff68e0446d2e47968f5200a86b558285" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35545 } health_report { overall_health: UNKNOWN } } }
W20260811 15:10:10.619488 17264 tablet.cc:2369] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:10:10.649405 17282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:10.643783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34250 (local address 127.30.8.3:41595)
0811 15:10:10.643923 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 15:10:10.643927 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:10:10.643943 (+ 16us) server_negotiation.cc:407] Connection header received
0811 15:10:10.644081 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:10:10.644084 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:10:10.644173 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:10:10.644297 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:10:10.644718 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.645511 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.646838 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:10.647236 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:10.647348 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:10.647861 (+ 513us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:10:10.648531 (+ 670us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:10:10.648652 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 15:10:10.649147 (+ 495us) server_negotiation.cc:299] Negotiation successful
0811 15:10:10.649222 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 15:10:10.661659 16783 raft_consensus.cc:1261] T 2addf7d5c3584027be0340e7ddaec144 P ff68e0446d2e47968f5200a86b558285 [term 1 FOLLOWER]: Refusing update from remote peer 6d5dc2aec08f4284ac0766eaed78c268: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:10:10.662762 17308 consensus_queue.cc:1035] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff68e0446d2e47968f5200a86b558285" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 35545 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:10:10.671587 16996 raft_consensus.cc:1261] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5 [term 1 FOLLOWER]: Refusing update from remote peer 6d5dc2aec08f4284ac0766eaed78c268: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:10:10.672338 17308 consensus_queue.cc:1035] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2af2d5f45d9f4add9422d8b9023343a5" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 36929 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 15:10:10.680806 17060 tablet.cc:2369] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:10:10.686463 17317 mvcc.cc:205] Tried to move back new op lower bound from 7317344299657998336 to 7317344299418042368. Current Snapshot: MvccSnapshot[applied={T|T < 7317344299657998336}]
I20260811 15:10:10.709643 16502 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:10:10.709973 16502 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57104:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:10:10.714233 16502 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 473c1cba35dc489fb37cf9051b172362: Sending DeleteTablet for 3 replicas of tablet 2addf7d5c3584027be0340e7ddaec144
I20260811 15:10:10.716212 17163 tablet_service.cc:1514] Processing DeleteTablet for tablet 2addf7d5c3584027be0340e7ddaec144 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:10:10 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:34236
I20260811 15:10:10.716353 16722 tablet_service.cc:1514] Processing DeleteTablet for tablet 2addf7d5c3584027be0340e7ddaec144 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:10:10 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:52980
I20260811 15:10:10.717814 16968 tablet_service.cc:1514] Processing DeleteTablet for tablet 2addf7d5c3584027be0340e7ddaec144 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:10:10 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:41518
I20260811 15:10:10.718690 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 16635
I20260811 15:10:10.735881 17340 tablet_replica.cc:330] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5: stopping tablet replica
I20260811 15:10:10.736498 17340 raft_consensus.cc:2229] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:10:10.737057 17340 raft_consensus.cc:2258] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:10:10.745733 17338 tablet_replica.cc:330] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268: stopping tablet replica
I20260811 15:10:10.746398 17338 raft_consensus.cc:2229] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:10:10.747502 17338 raft_consensus.cc:2258] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:10:10.748657 17340 ts_tablet_manager.cc:1905] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:10:10.750603 17338 ts_tablet_manager.cc:1905] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:10:10.753165 17340 ts_tablet_manager.cc:1918] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:10:10.753317 17340 log.cc:1196] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/2addf7d5c3584027be0340e7ddaec144
I20260811 15:10:10.753677 17340 ts_tablet_manager.cc:1939] T 2addf7d5c3584027be0340e7ddaec144 P 2af2d5f45d9f4add9422d8b9023343a5: Deleting consensus metadata
I20260811 15:10:10.754906 17338 ts_tablet_manager.cc:1918] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:10:10.755179 17338 log.cc:1196] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/2addf7d5c3584027be0340e7ddaec144
I20260811 15:10:10.755658 17338 ts_tablet_manager.cc:1939] T 2addf7d5c3584027be0340e7ddaec144 P 6d5dc2aec08f4284ac0766eaed78c268: Deleting consensus metadata
I20260811 15:10:10.757503 16478 catalog_manager.cc:4841] TS 6d5dc2aec08f4284ac0766eaed78c268 (127.30.8.3:41595): tablet 2addf7d5c3584027be0340e7ddaec144 (table test-table [id=90d8eddb64fe4f50843811ae2c6c7887]) successfully deleted
W20260811 15:10:10.768903 16480 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.8.1:35545 (error 108)
I20260811 15:10:10.770433 16479 catalog_manager.cc:4841] TS 2af2d5f45d9f4add9422d8b9023343a5 (127.30.8.2:36929): tablet 2addf7d5c3584027be0340e7ddaec144 (table test-table [id=90d8eddb64fe4f50843811ae2c6c7887]) successfully deleted
I20260811 15:10:10.771202 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 16862
I20260811 15:10:10.786953 17284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:10.786542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:35545 (local address 127.0.0.1:52990)
0811 15:10:10.786719 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 15:10:10.786817 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.1:35545: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 15:10:10.787329 16480 catalog_manager.cc:4584] TS ff68e0446d2e47968f5200a86b558285 (127.30.8.1:35545): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2addf7d5c3584027be0340e7ddaec144: Network error: Client connection negotiation failed: client connection to 127.30.8.1:35545: connect: Connection refused (error 111)
I20260811 15:10:10.793690 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 17066
I20260811 15:10:10.828933 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 16457
I20260811 15:10:10.845202 30752 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 15:10:11.192878 30752 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 15:10:11.193030 30752 mini_ranger.cc:66] Stopping Ranger...
I20260811 15:10:11.539074 30752 mini_ranger.cc:68] Stopped Ranger
2026-08-11 15:10:11.539 UTC [15808] LOG: received smart shutdown request
2026-08-11 15:10:11.544 UTC [15808] LOG: background worker "logical replication launcher" (PID 15821) exited with exit code 1
2026-08-11 15:10:11.545 UTC [15816] LOG: shutting down
2026-08-11 15:10:11.743 UTC [15808] LOG: database system is shut down
2026-08-11T15:10:11Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (49620 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:10:11 dist-test-slave-1vmg krb5kdc[17362](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:10:11 dist-test-slave-1vmg krb5kdc[17362](info): set up 2 sockets
Aug 11 15:10:11 dist-test-slave-1vmg krb5kdc[17362](info): commencing operation
krb5kdc: starting...
W20260811 15:10:13.049571 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.166s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:10:13 dist-test-slave-1vmg krb5kdc[17362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461013, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:10:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:10:13Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.8.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.8.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.8.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/rangeradmin_127.30.8.20@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.8.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/rangeradmin_127.30.8.20@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.8.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.8.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.8.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/rangerlookup_127.30.8.20@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.8.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/rangerlookup_127.30.8.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.8.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.8.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.8.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/HTTP_127.30.8.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.8.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/HTTP_127.30.8.20@KRBTEST.COM.keytab.
I20260811 15:10:13.254088 30752 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskxgyPSN/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/postgres -l logfile start
2026-08-11 15:10:15.818 UTC [17400] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 15:10:15.818 UTC [17400] LOG: listening on IPv4 address "127.30.8.20", port 40641
2026-08-11 15:10:15.822 UTC [17400] LOG: listening on Unix socket "/tmp/.s.PGSQL.40641"
2026-08-11 15:10:15.850 UTC [17404] LOG: database system was shut down at 2026-08-11 15:10:14 UTC
2026-08-11 15:10:15.853 UTC [17400] LOG: database system is ready to accept connections
I20260811 15:10:16.958367 30752 mini_postgres.cc:96] Postgres bound to 40641
2026-08-11 15:10:16.966 UTC [17414] FATAL: database "slave" does not exist
127.30.8.20:40641 - accepting connections
I20260811 15:10:16.966820 30752 mini_ranger.cc:161] Starting Ranger...
I20260811 15:10:16.983605 30752 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 15:10:17.565076 30752 mini_ranger.cc:87] Created ranger Postgres database
I20260811 15:10:17.565157 30752 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ranger-admin
2026-08-11 15:10:17,867 [I] DB FLAVOR :POSTGRES
2026-08-11 15:10:17,867 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 15:10:17,867 [I] Checking connection..
2026-08-11 15:10:17,867 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 15:10:18,151 [I] Checking connection passed.
2026-08-11 15:10:18,151 [I] --------- Verifying version history table ---------
2026-08-11 15:10:18,151 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 15:10:18,417 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 15:10:18,418 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 15:10:18,684 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 15:10:18,684 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 15:10:18,685 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 15:10:18.950 UTC [17520] WARNING: there is no transaction in progress
2026-08-11 15:10:18,957 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 15:10:18,957 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 15:10:19,219 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 15:10:19,219 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 15:10:19,219 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 15:10:19,486 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 15:10:19,749 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-1vmg.c.gcp-upstream.internal','N') ;"
2026-08-11 15:10:20,037 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 15:10:20,309 [I] Table x_portal_user does not exist in database ranger
2026-08-11 15:10:20,310 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 15:10:20,604 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 15:10:20,604 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 15:10:20,881 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 15:10:21,157 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 15:10:21,157 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 15:10:22.228 UTC [17720] WARNING: there is no transaction in progress
2026-08-11 15:10:22.250 UTC [17720] WARNING: there is no transaction in progress
2026-08-11 15:10:22.272 UTC [17720] WARNING: there is no transaction in progress
2026-08-11 15:10:22.293 UTC [17720] WARNING: there is no transaction in progress
2026-08-11 15:10:22.315 UTC [17720] WARNING: there is no transaction in progress
2026-08-11 15:10:22.338 UTC [17720] WARNING: there is no transaction in progress
2026-08-11 15:10:22.405 UTC [17720] WARNING: there is no transaction in progress
2026-08-11 15:10:22.413 UTC [17720] WARNING: there is no transaction in progress
2026-08-11 15:10:22.430 UTC [17720] WARNING: there is no transaction in progress
2026-08-11 15:10:22.441 UTC [17720] WARNING: there is no transaction in progress
2026-08-11 15:10:22.453 UTC [17720] WARNING: there is no transaction in progress
2026-08-11 15:10:22.470 UTC [17720] WARNING: there is no transaction in progress
2026-08-11 15:10:22.482 UTC [17720] WARNING: there is no transaction in progress
2026-08-11 15:10:22.489 UTC [17720] WARNING: there is no transaction in progress
2026-08-11 15:10:22.499 UTC [17720] WARNING: there is no transaction in progress
2026-08-11 15:10:22.507 UTC [17720] WARNING: there is no transaction in progress
2026-08-11 15:10:23,294 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 15:10:23,295 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 15:10:23,567 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 15:10:23,568 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 15:10:23,983 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 15:10:23,983 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 15:10:24,455 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 15:10:24,456 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 15:10:24,724 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 15:10:25,011 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-1vmg.c.gcp-upstream.internal';"
2026-08-11 15:10:25,269 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 15:10:25,269 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 15:10:25,547 [I] DB_PATCHES have already been applied
I20260811 15:10:25.553125 30752 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ranger-admin:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskxgyPSN/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 15:10:25.553210 30752 mini_ranger.cc:193] Using host: 127.30.8.20
I20260811 15:10:25.559448 30752 mini_ranger.cc:227] Ranger admin URL: http://127.30.8.20:35949
Aug 11, 2026 3:10:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 3:10:25 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 3:10:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 3:10:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 3:10:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 3:10:26 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 3:10:26 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 15:10:26.843106 30752 mini_ranger.cc:160] Time spent starting Ranger: real 9.876s user 0.000s sys 0.014s
Aug 11, 2026 3:10:48 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 3:10:48 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 3:10:48 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 3:10:48 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 3:10:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 3:10:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 3:10:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 3:10:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 3:10:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 3:10:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 3:10:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 3:10:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 3:10:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 3:10:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 3:10:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 3:10:48 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 3:10:48 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 3:10:49 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 15:10:49.741061 30752 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.8.20@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.8.20@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.8.20@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/rangerkms_127.30.8.20@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.8.20@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/rangerkms_127.30.8.20@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.8.20@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.8.20@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.8.20@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/HTTP_127.30.8.20@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.8.20@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/HTTP_127.30.8.20@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 15:10:49 dist-test-slave-1vmg krb5kdc[17362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461049, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 15:10:49.839620 30752 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 15:10:49.851562 30752 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 15:10:50.629081 30752 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 15:10:50.629197 30752 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ranger-kms
I20260811 15:10:50.629215 30752 mini_ranger_kms.cc:227] Using postgres at 127.30.8.20:40641
2026-08-11 15:10:50,795 [I] DB FLAVOR :POSTGRES
2026-08-11 15:10:50,796 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 15:10:50,796 [I] Checking connection
2026-08-11 15:10:50,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 15:10:51,063 [I] connection success
2026-08-11 15:10:51,064 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 15:10:51,064 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 15:10:51,064 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 15:10:51,304 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 15:10:51,304 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 15:10:51,304 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 15:10:51,305 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.8.20:40641/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 15:10:51,584 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 15:10:52.131968 30752 mini_ranger_kms.cc:310] Created kms service
I20260811 15:10:52.345551 30752 mini_ranger_kms.cc:326] Created kudu user
I20260811 15:10:52.393333 30752 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 15:10:52 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461049, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
I20260811 15:10:52.792685 30752 mini_ranger_kms.cc:384] Added ranger policy
I20260811 15:10:52.792958 30752 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ranger-kms:/tmp/dist-test-taskxgyPSN/build/debug/bin/postgresql.jar:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskxgyPSN/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 15:10:52.792990 30752 mini_ranger_kms.cc:240] Using host: 127.30.8.20
I20260811 15:10:52.797840 30752 mini_ranger_kms.cc:276] Ranger KMS PID: 18040
I20260811 15:10:52.797940 30752 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.8.20:57211
Aug 11, 2026 3:10:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 3:10:53 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 3:10:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 3:10:53 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 3:10:54 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 3:10:54 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 15:10:55 dist-test-slave-1vmg krb5kdc[17362](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786461055, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.8.20@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:10:55.360441 30752 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.521s user 0.025s sys 0.128s
I20260811 15:10:55.360558 30752 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 15:10:55.360631 30752 mini_ranger_kms.cc:401] 127.30.8.20:57211/kms/v1/keys
Aug 11 15:10:55 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461055, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.8.20@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
Aug 11 15:10:56 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461055, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.8.20@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
Aug 11 15:10:58 dist-test-slave-1vmg krb5kdc[17362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:10:58.402976 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:43447
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:43447
--ranger_config_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.8.20:57211/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:10:58.525563 18116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:10:58.525877 18116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:10:58.525986 18116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:10:58.530249 18116 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:10:58.530325 18116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:10:58.530344 18116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:10:58.530375 18116 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:10:58.530403 18116 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:10:58.536374 18116 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.8.20:57211/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:43447
--ranger_config_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:43447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18116
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:10:58.537582 18116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:10:58.538650 18116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:10:58.547472 18126 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:10:58.547557 18132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:10:58.547489 18129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:10:58.547536 18116 server_base.cc:1034] running on GCE node
I20260811 15:10:58.548275 18116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:10:58.549489 18116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:10:58.550683 18116 hybrid_clock.cc:648] HybridClock initialized: now 1786461058550669 us; error 28 us; skew 500 ppm
Aug 11 15:10:58 dist-test-slave-1vmg krb5kdc[17362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461058, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:10:58.553494 18116 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:10:58.554495 18116 webserver.cc:460] Webserver started at http://127.30.8.62:36957/ using document root <none> and password file <none>
I20260811 15:10:58.555114 18116 fs_manager.cc:359] Metadata directory not provided
I20260811 15:10:58.555176 18116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:10:58.555408 18116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 15:10:58 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461058, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
I20260811 15:10:58.613713 18116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "ff319b1a69c24cd397beded0cfbaae75"
format_stamp: "Formatted at 2026-08-11 15:10:58 on dist-test-slave-1vmg"
server_key: "0dad8575219a7958153e44aec593fb97"
server_key_iv: "f3289e486bdb973df424a06bd900cb6b"
server_key_version: "kuduclusterkey@0"
I20260811 15:10:58.614535 18116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "ff319b1a69c24cd397beded0cfbaae75"
format_stamp: "Formatted at 2026-08-11 15:10:58 on dist-test-slave-1vmg"
server_key: "0dad8575219a7958153e44aec593fb97"
server_key_iv: "f3289e486bdb973df424a06bd900cb6b"
server_key_version: "kuduclusterkey@0"
I20260811 15:10:58.620352 18116 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260811 15:10:58.641212 18143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:10:58.643036 18116 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:10:58.643200 18116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "ff319b1a69c24cd397beded0cfbaae75"
format_stamp: "Formatted at 2026-08-11 15:10:58 on dist-test-slave-1vmg"
server_key: "0dad8575219a7958153e44aec593fb97"
server_key_iv: "f3289e486bdb973df424a06bd900cb6b"
server_key_version: "kuduclusterkey@0"
I20260811 15:10:58.643350 18116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:10:58.664644 18116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:10:58.668020 18116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:10:58.668241 18116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:10:58.677977 18222 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:43447 every 8 connection(s)
I20260811 15:10:58.677986 18116 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:43447
I20260811 15:10:58.679144 18116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:10:58.682574 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 18116
I20260811 15:10:58.682688 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:10:58.682876 18223 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:10:58.683032 30752 external_mini_cluster.cc:1428] Setting key 2787af5f0bb053723f146e84efb9d1bd
I20260811 15:10:58.691009 18223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75: Bootstrap starting.
Aug 11 15:10:58 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461058, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:10:58.693990 18223 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75: Neither blocks nor log segments found. Creating new log.
I20260811 15:10:58.695451 18223 log.cc:824] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75: Log is configured to *not* fsync() on all Append() calls
I20260811 15:10:58.697405 18229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:58.684500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41614 (local address 127.30.8.62:43447)
0811 15:10:58.684931 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0811 15:10:58.684954 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 15:10:58.684990 (+ 36us) server_negotiation.cc:407] Connection header received
0811 15:10:58.685923 (+ 933us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:10:58.685942 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:10:58.686308 (+ 366us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:10:58.686631 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:10:58.687514 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:58.688519 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:58.689371 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:58.689853 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:58.690004 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:58.692589 (+ 2585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:10:58.692595 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:10:58.692610 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:10:58.692641 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:10:58.694231 (+ 1590us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:10:58.694660 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:10:58.694666 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:10:58.694671 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:10:58.694790 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:10:58.695106 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:10:58.695109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:10:58.695110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:10:58.695553 (+ 443us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:10:58.695789 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 15:10:58.696105 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 15:10:58.696427 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":122,"threads_started":1}
I20260811 15:10:58.697989 18223 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75: No bootstrap required, opened a new log
I20260811 15:10:58.701406 18223 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff319b1a69c24cd397beded0cfbaae75" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43447 } }
I20260811 15:10:58.701658 18223 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:10:58.701696 18223 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff319b1a69c24cd397beded0cfbaae75, State: Initialized, Role: FOLLOWER
I20260811 15:10:58.702148 18223 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff319b1a69c24cd397beded0cfbaae75" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43447 } }
I20260811 15:10:58.702276 18223 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:10:58.702337 18223 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:10:58.702435 18223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:10:58.703487 18223 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff319b1a69c24cd397beded0cfbaae75" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43447 } }
I20260811 15:10:58.703860 18223 leader_election.cc:302] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff319b1a69c24cd397beded0cfbaae75; no voters:
I20260811 15:10:58.704219 18223 leader_election.cc:288] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:10:58.704361 18232 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:10:58.704620 18232 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [term 1 LEADER]: Becoming Leader. State: Replica: ff319b1a69c24cd397beded0cfbaae75, State: Running, Role: LEADER
I20260811 15:10:58.704955 18232 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff319b1a69c24cd397beded0cfbaae75" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43447 } }
I20260811 15:10:58.705693 18223 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:10:58.706370 18232 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff319b1a69c24cd397beded0cfbaae75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff319b1a69c24cd397beded0cfbaae75" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43447 } } }
I20260811 15:10:58.706487 18232 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [sys.catalog]: This master's current role is: LEADER
I20260811 15:10:58.706713 18232 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff319b1a69c24cd397beded0cfbaae75. Latest consensus state: current_term: 1 leader_uuid: "ff319b1a69c24cd397beded0cfbaae75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff319b1a69c24cd397beded0cfbaae75" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43447 } } }
I20260811 15:10:58.706791 18232 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [sys.catalog]: This master's current role is: LEADER
I20260811 15:10:58.707088 18254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:10:58.710605 18254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:10:58.711196 18223 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 15:10:58.724555 18254 catalog_manager.cc:1347] Generated new cluster ID: 860d97be5cc14ed7be5e81792cbe0aba
I20260811 15:10:58.724656 18254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:10:58.738258 18254 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:10:58.739816 18254 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:10:58.755616 18254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75: Generated new TSK 0
I20260811 15:10:58.756708 18254 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
I20260811 15:10:59.451640 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.8.20:57211/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:43447
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:10:59.608989 18320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:10:59.609388 18320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:10:59.609445 18320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:10:59.613761 18320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:10:59.613833 18320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:10:59.613898 18320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:10:59.622035 18320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.8.20:57211/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:43447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:10:59.623553 18320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:10:59.624872 18320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:10:59.636658 18340 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:10:59.636940 18335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:10:59.637183 18336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:10:59.637440 18320 server_base.cc:1034] running on GCE node
I20260811 15:10:59.637962 18320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:10:59.638823 18320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:10:59.640017 18320 hybrid_clock.cc:648] HybridClock initialized: now 1786461059639987 us; error 47 us; skew 500 ppm
Aug 11 15:10:59 dist-test-slave-1vmg krb5kdc[17362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461059, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:10:59.643826 18320 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:10:59.645139 18320 webserver.cc:460] Webserver started at http://127.30.8.1:34151/ using document root <none> and password file <none>
I20260811 15:10:59.645952 18320 fs_manager.cc:359] Metadata directory not provided
I20260811 15:10:59.646062 18320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:10:59.646343 18320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 15:10:59 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461059, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
I20260811 15:10:59.664322 18320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "3b9df95c364c4460a5a97e9045211b52"
format_stamp: "Formatted at 2026-08-11 15:10:59 on dist-test-slave-1vmg"
server_key: "78be3b9eeba78d50243c7ee21b0a10da"
server_key_iv: "a48e8ccebd42c9c5c8c94741033e4ef6"
server_key_version: "kuduclusterkey@0"
I20260811 15:10:59.665182 18320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "3b9df95c364c4460a5a97e9045211b52"
format_stamp: "Formatted at 2026-08-11 15:10:59 on dist-test-slave-1vmg"
server_key: "78be3b9eeba78d50243c7ee21b0a10da"
server_key_iv: "a48e8ccebd42c9c5c8c94741033e4ef6"
server_key_version: "kuduclusterkey@0"
I20260811 15:10:59.684048 18320 fs_manager.cc:691] Time spent creating directory manager: real 0.018s user 0.006s sys 0.000s
I20260811 15:10:59.711580 18351 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:10:59.713708 18320 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.004s sys 0.000s
I20260811 15:10:59.713860 18320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "3b9df95c364c4460a5a97e9045211b52"
format_stamp: "Formatted at 2026-08-11 15:10:59 on dist-test-slave-1vmg"
server_key: "78be3b9eeba78d50243c7ee21b0a10da"
server_key_iv: "a48e8ccebd42c9c5c8c94741033e4ef6"
server_key_version: "kuduclusterkey@0"
I20260811 15:10:59.713995 18320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:10:59.760406 18320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:10:59.763581 18320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:10:59.763782 18320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:10:59.764430 18320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:10:59.765380 18320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:10:59.765432 18320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:10:59.765488 18320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:10:59.765544 18320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:10:59.777285 18320 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:36817
I20260811 15:10:59.777281 18543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:36817 every 8 connection(s)
I20260811 15:10:59.778656 18320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:10:59.780957 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 18320
I20260811 15:10:59.781083 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:10:59.781394 30752 external_mini_cluster.cc:1428] Setting key 529411b4c18da77a0e1654c831203af0
Aug 11 15:10:59 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461059, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.8.2@KRBTEST.COM; defaulting to no policy
I20260811 15:10:59.797137 18549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:59.781206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:34323 (local address 127.30.8.62:43447)
0811 15:10:59.781577 (+ 371us) server_negotiation.cc:206] Beginning negotiation
0811 15:10:59.781582 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:10:59.784459 (+ 2877us) server_negotiation.cc:407] Connection header received
0811 15:10:59.784508 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:10:59.784511 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:10:59.784568 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:10:59.784645 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:10:59.786094 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:59.786949 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:59.788391 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:10:59.788730 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:10:59.789495 (+ 765us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:59.791499 (+ 2004us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:10:59.791502 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:10:59.791504 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:10:59.791530 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:10:59.793370 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:10:59.794092 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:10:59.794095 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:10:59.794097 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:10:59.794197 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:10:59.794674 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:10:59.794677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:10:59.794679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:10:59.794978 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:10:59.796757 (+ 1779us) server_negotiation.cc:1035] Waiting for connection context
0811 15:10:59.796839 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 15:10:59.796980 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":276,"thread_start_us":97,"threads_started":1}
I20260811 15:10:59.797374 18551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:10:59.781657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:43447 (local address 127.30.8.1:34323)
0811 15:10:59.783073 (+ 1416us) negotiation.cc:107] Waiting for socket to connect
0811 15:10:59.783137 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 15:10:59.784152 (+ 1015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:10:59.784979 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:10:59.784988 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:10:59.785379 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:10:59.785934 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:10:59.785949 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:10:59.787108 (+ 1159us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:10:59.787112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:10:59.788256 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:10:59.788267 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:10:59.788951 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:10:59.788955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:10:59.789163 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:10:59.789847 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:10:59.789865 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:10:59.791307 (+ 1442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:10:59.793523 (+ 2216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:10:59.793532 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:10:59.793554 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:10:59.793944 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:10:59.794309 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:10:59.794312 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:10:59.794314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:10:59.794537 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:10:59.795130 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:10:59.795137 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:10:59.795548 (+ 411us) client_negotiation.cc:769] Sending connection context
0811 15:10:59.795784 (+ 236us) client_negotiation.cc:240] Negotiation successful
0811 15:10:59.796185 (+ 401us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1124,"thread_start_us":281,"threads_started":1}
I20260811 15:10:59.799013 18547 heartbeater.cc:347] Connected to a master server at 127.30.8.62:43447
I20260811 15:10:59.799289 18547 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.2@KRBTEST.COM" created.
I20260811 15:10:59.799944 18547 heartbeater.cc:511] Master 127.30.8.62:43447 requested a full tablet report, sending...
I20260811 15:10:59.801759 18171 ts_manager.cc:194] Registered new tserver with Master: 3b9df95c364c4460a5a97e9045211b52 (127.30.8.1:36817)
I20260811 15:10:59.803316 18171 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:34323
Aug 11 15:10:59 dist-test-slave-1vmg krb5kdc[17362](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786461059, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
I20260811 15:10:59.818161 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.8.20:57211/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:43447
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:10:59.953749 18557 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:10:59.954061 18557 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:10:59.954128 18557 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:10:59.959018 18557 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:10:59.959100 18557 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:10:59.959185 18557 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:10:59.966509 18557 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.8.20:57211/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:43447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18557
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:10:59.967772 18557 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:10:59.968869 18557 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:10:59.980113 18566 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:10:59.980234 18564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:10:59.980119 18568 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:10:59.981130 18557 server_base.cc:1034] running on GCE node
I20260811 15:10:59.981590 18557 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:10:59.982302 18557 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:10:59.983476 18557 hybrid_clock.cc:648] HybridClock initialized: now 1786461059983425 us; error 88 us; skew 500 ppm
Aug 11 15:10:59 dist-test-slave-1vmg krb5kdc[17362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461059, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:10:59.986899 18557 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:10:59.988166 18557 webserver.cc:460] Webserver started at http://127.30.8.2:34547/ using document root <none> and password file <none>
I20260811 15:10:59.988904 18557 fs_manager.cc:359] Metadata directory not provided
I20260811 15:10:59.988981 18557 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:10:59.989251 18557 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 15:10:59 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461059, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
I20260811 15:11:00.017490 18557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "8986648b7c4b42fa8f10976589b80188"
format_stamp: "Formatted at 2026-08-11 15:11:00 on dist-test-slave-1vmg"
server_key: "5b002fc6bec84bf6de08215489ce40f6"
server_key_iv: "694fc47667bda8b233d65c32271e0757"
server_key_version: "kuduclusterkey@0"
I20260811 15:11:00.018441 18557 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "8986648b7c4b42fa8f10976589b80188"
format_stamp: "Formatted at 2026-08-11 15:11:00 on dist-test-slave-1vmg"
server_key: "5b002fc6bec84bf6de08215489ce40f6"
server_key_iv: "694fc47667bda8b233d65c32271e0757"
server_key_version: "kuduclusterkey@0"
I20260811 15:11:00.025689 18557 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260811 15:11:00.049510 18579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:00.051433 18557 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 15:11:00.051688 18557 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "8986648b7c4b42fa8f10976589b80188"
format_stamp: "Formatted at 2026-08-11 15:11:00 on dist-test-slave-1vmg"
server_key: "5b002fc6bec84bf6de08215489ce40f6"
server_key_iv: "694fc47667bda8b233d65c32271e0757"
server_key_version: "kuduclusterkey@0"
I20260811 15:11:00.051959 18557 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:00.070585 18557 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:00.075062 18557 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:00.075382 18557 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:00.076323 18557 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:11:00.077625 18557 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:11:00.077766 18557 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:00.077893 18557 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:11:00.077970 18557 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:00.094671 18557 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:42983
I20260811 15:11:00.096233 18557 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:11:00.104866 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 18557
I20260811 15:11:00.105027 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:11:00.105450 30752 external_mini_cluster.cc:1428] Setting key 712a05ec94e261dcf4220b7ea3e46adc
I20260811 15:11:00.099648 18783 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:42983 every 8 connection(s)
Aug 11 15:11:00 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461059, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:00.124310 18549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:00.104468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:58111 (local address 127.30.8.62:43447)
0811 15:11:00.104725 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:00.104731 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:00.105481 (+ 750us) server_negotiation.cc:407] Connection header received
0811 15:11:00.107443 (+ 1962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:00.107448 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:00.107514 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:00.107574 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:00.109675 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:00.110579 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:00.112584 (+ 2005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:00.112930 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:00.113019 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:00.119046 (+ 6027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:00.119052 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:00.119055 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:00.119098 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:00.121107 (+ 2009us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:00.121911 (+ 804us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:00.121915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:00.121917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:00.122021 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:00.122521 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:00.122525 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:00.122527 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:00.122820 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:00.122952 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:00.123870 (+ 918us) server_negotiation.cc:299] Negotiation successful
0811 15:11:00.124054 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260811 15:11:00.126868 18786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:00.102965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:43447 (local address 127.30.8.2:58111)
0811 15:11:00.105258 (+ 2293us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:00.105342 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:00.107122 (+ 1780us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:00.108102 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:00.108113 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:00.108597 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:00.109461 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:00.109488 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:00.110773 (+ 1285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:00.110777 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:00.112421 (+ 1644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:00.112435 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:00.113282 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:00.113286 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:00.113529 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:00.116281 (+ 2752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:00.116308 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:00.118709 (+ 2401us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:00.121282 (+ 2573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:00.121290 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:00.121315 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:00.121783 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:00.122140 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:00.122144 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:00.122146 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:00.122393 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:00.123029 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:00.123038 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:00.123514 (+ 476us) client_negotiation.cc:769] Sending connection context
0811 15:11:00.124971 (+ 1457us) client_negotiation.cc:240] Negotiation successful
0811 15:11:00.125451 (+ 480us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1777,"thread_start_us":239,"threads_started":1}
WARNING: no policy specified for kudu/127.30.8.3@KRBTEST.COM; defaulting to no policy
I20260811 15:11:00.129133 18784 heartbeater.cc:347] Connected to a master server at 127.30.8.62:43447
I20260811 15:11:00.129495 18784 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:00.130179 18784 heartbeater.cc:511] Master 127.30.8.62:43447 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.3@KRBTEST.COM" created.
I20260811 15:11:00.133936 18171 ts_manager.cc:194] Registered new tserver with Master: 8986648b7c4b42fa8f10976589b80188 (127.30.8.2:42983)
I20260811 15:11:00.134756 18171 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:58111
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
I20260811 15:11:00.148351 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.8.20:57211/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:43447
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:11:00.319193 18793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:00.320055 18793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:00.320185 18793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:00.326296 18793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:00.326475 18793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:00.326612 18793 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:11:00.334901 18793 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.8.20:57211/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:43447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:00.336648 18793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:00.338074 18793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:00.351926 18799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:00.352159 18793 server_base.cc:1034] running on GCE node
W20260811 15:11:00.352492 18804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:00.352792 18800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:00.353386 18793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:00.354302 18793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:00.355602 18793 hybrid_clock.cc:648] HybridClock initialized: now 1786461060355529 us; error 92 us; skew 500 ppm
Aug 11 15:11:00 dist-test-slave-1vmg krb5kdc[17362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:00.359712 18793 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:11:00.361039 18793 webserver.cc:460] Webserver started at http://127.30.8.3:35047/ using document root <none> and password file <none>
I20260811 15:11:00.361860 18793 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:00.362015 18793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:00.362334 18793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 15:11:00 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
I20260811 15:11:00.390794 18793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "57d39dea94fc4c03bf550aade1632d07"
format_stamp: "Formatted at 2026-08-11 15:11:00 on dist-test-slave-1vmg"
server_key: "e863a5fb5f10c995dabc7a4bd1eeefc9"
server_key_iv: "6f54d39092425f54aace792c2e61f42a"
server_key_version: "kuduclusterkey@0"
I20260811 15:11:00.392200 18793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "57d39dea94fc4c03bf550aade1632d07"
format_stamp: "Formatted at 2026-08-11 15:11:00 on dist-test-slave-1vmg"
server_key: "e863a5fb5f10c995dabc7a4bd1eeefc9"
server_key_iv: "6f54d39092425f54aace792c2e61f42a"
server_key_version: "kuduclusterkey@0"
I20260811 15:11:00.397897 18793 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
Aug 11 15:11:00 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461059, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
I20260811 15:11:00.421883 18814 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:00.423609 18793 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 15:11:00.423744 18793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "57d39dea94fc4c03bf550aade1632d07"
format_stamp: "Formatted at 2026-08-11 15:11:00 on dist-test-slave-1vmg"
server_key: "e863a5fb5f10c995dabc7a4bd1eeefc9"
server_key_iv: "6f54d39092425f54aace792c2e61f42a"
server_key_version: "kuduclusterkey@0"
I20260811 15:11:00.423882 18793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:00.465893 18793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:00.469867 18793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:00.470146 18793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:00.470868 18793 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:11:00.472033 18793 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:11:00.472158 18793 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:00.472266 18793 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:11:00.472337 18793 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:00.485281 18793 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:38777
I20260811 15:11:00.489007 18793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:11:00.491487 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 18793
I20260811 15:11:00.491628 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:11:00.491948 30752 external_mini_cluster.cc:1428] Setting key c2498fd1753ae3bff0965061fbc4c5e3
I20260811 15:11:00.491011 19011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:38777 every 8 connection(s)
Aug 11 15:11:00 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:00.534485 18549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:00.495592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:50099 (local address 127.30.8.62:43447)
0811 15:11:00.495744 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:00.495750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:00.497734 (+ 1984us) server_negotiation.cc:407] Connection header received
0811 15:11:00.499188 (+ 1454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:00.499194 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:00.499261 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:00.499318 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:00.501449 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:00.502493 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:00.504255 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:00.504609 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:00.504689 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:00.527768 (+ 23079us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:00.527773 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:00.527776 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:00.527829 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:00.530008 (+ 2179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:00.530765 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:00.530768 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:00.530770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:00.530862 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:00.533022 (+ 2160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:00.533026 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:00.533028 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:00.533289 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:00.533369 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:00.534168 (+ 799us) server_negotiation.cc:299] Negotiation successful
0811 15:11:00.534309 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 15:11:00.536545 19016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:00.496182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:43447 (local address 127.30.8.3:50099)
0811 15:11:00.497558 (+ 1376us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:00.497623 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:00.498819 (+ 1196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:00.499855 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:00.499867 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:00.500420 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:00.501250 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:00.501276 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:00.502657 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:00.502661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:00.504112 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:00.504124 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:00.521027 (+ 16903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:00.521032 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:00.521359 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:00.522166 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:00.522190 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:00.527471 (+ 5281us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:00.530212 (+ 2741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:00.530219 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:00.530242 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:00.530640 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:00.532654 (+ 2014us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:00.532658 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:00.532660 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:00.532910 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:00.533447 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:00.533455 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:00.533860 (+ 405us) client_negotiation.cc:769] Sending connection context
0811 15:11:00.535000 (+ 1140us) client_negotiation.cc:240] Negotiation successful
0811 15:11:00.535416 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1133,"thread_start_us":195,"threads_started":1}
I20260811 15:11:00.538210 19012 heartbeater.cc:347] Connected to a master server at 127.30.8.62:43447
I20260811 15:11:00.538471 19012 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:00.539140 19012 heartbeater.cc:511] Master 127.30.8.62:43447 requested a full tablet report, sending...
I20260811 15:11:00.540428 18171 ts_manager.cc:194] Registered new tserver with Master: 57d39dea94fc4c03bf550aade1632d07 (127.30.8.3:38777)
I20260811 15:11:00.541190 18171 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:50099
I20260811 15:11:00.544480 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 15:11:00.545334 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 18320
I20260811 15:11:00.573045 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 18557
I20260811 15:11:00.590013 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 18793
I20260811 15:11:00.622486 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 18116
I20260811 15:11:00.639777 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:43447
--webserver_interface=127.30.8.62
--webserver_port=36957
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:43447
--ranger_config_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.8.20:57211/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.8.20:57211/kms with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:11:00.757966 19018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:00.758270 19018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:00.758314 19018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:00.762562 19018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:11:00.762651 19018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:00.762670 19018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:00.762686 19018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:11:00.762717 19018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:11:00.768669 19018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.8.20:57211/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:43447
--ranger_config_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:43447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=36957
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:00.769866 19018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:00.770968 19018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:00.779901 19025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:00.779958 19018 server_base.cc:1034] running on GCE node
W20260811 15:11:00.779908 19023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:00.779901 19030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:00.781122 19018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:00.782661 19018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:00.783859 19018 hybrid_clock.cc:648] HybridClock initialized: now 1786461060783822 us; error 80 us; skew 500 ppm
Aug 11 15:11:00 dist-test-slave-1vmg krb5kdc[17362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:00.786849 19018 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:11:00.787909 19018 webserver.cc:460] Webserver started at http://127.30.8.62:36957/ using document root <none> and password file <none>
I20260811 15:11:00.788481 19018 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:00.788555 19018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 15:11:00 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461060, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
I20260811 15:11:00.861819 19018 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:11:00.864264 19041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:00.865599 19018 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 15:11:00.865736 19018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "ff319b1a69c24cd397beded0cfbaae75"
format_stamp: "Formatted at 2026-08-11 15:10:58 on dist-test-slave-1vmg"
server_key: "0dad8575219a7958153e44aec593fb97"
server_key_iv: "f3289e486bdb973df424a06bd900cb6b"
server_key_version: "kuduclusterkey@0"
I20260811 15:11:00.866307 19018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:00.877367 19018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:00.880129 19018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:00.880322 19018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:00.887733 19018 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:43447
I20260811 15:11:00.887744 19132 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:43447 every 8 connection(s)
I20260811 15:11:00.888936 19018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:11:00.893474 19134 sys_catalog.cc:263] Verifying existing consensus state
I20260811 15:11:00.895314 19134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75: Bootstrap starting.
I20260811 15:11:00.899164 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 19018
I20260811 15:11:00.899586 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.8.20:57211/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:36817
--local_ip_for_outbound_sockets=127.30.8.1
--tserver_master_addrs=127.30.8.62:43447
--webserver_port=34151
--webserver_interface=127.30.8.1
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.8.20:57211/kms with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:11:00.912201 19134 log.cc:824] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:00.919343 19134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 15:11:00.919761 19134 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75: Bootstrap complete.
I20260811 15:11:00.923954 19134 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff319b1a69c24cd397beded0cfbaae75" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43447 } }
I20260811 15:11:00.924368 19134 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff319b1a69c24cd397beded0cfbaae75, State: Initialized, Role: FOLLOWER
I20260811 15:11:00.924928 19134 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff319b1a69c24cd397beded0cfbaae75" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43447 } }
I20260811 15:11:00.925066 19134 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:11:00.925137 19134 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:11:00.925235 19134 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [term 1 FOLLOWER]: Advancing to term 2
I20260811 15:11:00.927280 19134 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff319b1a69c24cd397beded0cfbaae75" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43447 } }
I20260811 15:11:00.927618 19134 leader_election.cc:302] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff319b1a69c24cd397beded0cfbaae75; no voters:
I20260811 15:11:00.927942 19134 leader_election.cc:288] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 15:11:00.928061 19139 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 15:11:00.928243 19139 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [term 2 LEADER]: Becoming Leader. State: Replica: ff319b1a69c24cd397beded0cfbaae75, State: Running, Role: LEADER
I20260811 15:11:00.928617 19139 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff319b1a69c24cd397beded0cfbaae75" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43447 } }
I20260811 15:11:00.929273 19134 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:11:00.930145 19141 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff319b1a69c24cd397beded0cfbaae75. Latest consensus state: current_term: 2 leader_uuid: "ff319b1a69c24cd397beded0cfbaae75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff319b1a69c24cd397beded0cfbaae75" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43447 } } }
I20260811 15:11:00.930276 19141 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [sys.catalog]: This master's current role is: LEADER
I20260811 15:11:00.930567 19140 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "ff319b1a69c24cd397beded0cfbaae75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff319b1a69c24cd397beded0cfbaae75" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43447 } } }
I20260811 15:11:00.930640 19152 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:11:00.930641 19140 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75 [sys.catalog]: This master's current role is: LEADER
I20260811 15:11:00.934278 19152 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:11:00.934859 19134 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 15:11:00.935081 19152 catalog_manager.cc:1259] Loaded cluster ID: 860d97be5cc14ed7be5e81792cbe0aba
I20260811 15:11:00.935120 19152 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:11:00.937034 19152 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:11:00.937592 19152 catalog_manager.cc:5879] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75: Loaded TSK: 0
I20260811 15:11:00.938547 19152 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 15:11:01.032449 19136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:01.032756 19136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:01.032820 19136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:01.037551 19136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:01.037628 19136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:01.037721 19136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:11:01.043798 19136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.8.20:57211/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:36817
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=34151
--tserver_master_addrs=127.30.8.62:43447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:01.045048 19136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:01.046137 19136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:01.059887 19193 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:01.060885 19195 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:01.061120 19136 server_base.cc:1034] running on GCE node
W20260811 15:11:01.060880 19189 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:01.061707 19136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:01.062407 19136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:01.063617 19136 hybrid_clock.cc:648] HybridClock initialized: now 1786461061063594 us; error 35 us; skew 500 ppm
Aug 11 15:11:01 dist-test-slave-1vmg krb5kdc[17362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:01.066594 19136 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:11:01.067713 19136 webserver.cc:460] Webserver started at http://127.30.8.1:34151/ using document root <none> and password file <none>
I20260811 15:11:01.068292 19136 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:01.068382 19136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 15:11:01 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
I20260811 15:11:01.099186 19136 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:11:01.102085 19208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:01.103506 19136 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:11:01.103653 19136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "3b9df95c364c4460a5a97e9045211b52"
format_stamp: "Formatted at 2026-08-11 15:10:59 on dist-test-slave-1vmg"
server_key: "78be3b9eeba78d50243c7ee21b0a10da"
server_key_iv: "a48e8ccebd42c9c5c8c94741033e4ef6"
server_key_version: "kuduclusterkey@0"
I20260811 15:11:01.105005 19136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:01.124933 19136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:01.128873 19136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:01.129175 19136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:01.130100 19136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:11:01.131259 19136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:11:01.131316 19136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:01.131372 19136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:11:01.131415 19136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:01.142436 19136 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:36817
I20260811 15:11:01.142608 19397 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:36817 every 8 connection(s)
I20260811 15:11:01.143859 19136 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:11:01.148396 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 19136
I20260811 15:11:01.148835 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.8.20:57211/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:42983
--local_ip_for_outbound_sockets=127.30.8.2
--tserver_master_addrs=127.30.8.62:43447
--webserver_port=34547
--webserver_interface=127.30.8.2
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.8.20:57211/kms with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 15:11:01 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:01.174623 19402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:01.150768 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:43447 (local address 127.30.8.1:35045)
0811 15:11:01.153746 (+ 2978us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:01.153815 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:01.155074 (+ 1259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:01.157605 (+ 2531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:01.157615 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:01.158064 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:01.158782 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:01.158808 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:01.161001 (+ 2193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:01.161006 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:01.162431 (+ 1425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:01.162447 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:01.163835 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:01.163839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:01.164037 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:01.164707 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:01.164731 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:01.166551 (+ 1820us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:01.170483 (+ 3932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:01.170491 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:01.170510 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:01.170896 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:01.171478 (+ 582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:01.171483 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:01.171485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:01.171742 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:01.172596 (+ 854us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:01.172603 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:01.172937 (+ 334us) client_negotiation.cc:769] Sending connection context
0811 15:11:01.173152 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 15:11:01.173523 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2695,"thread_start_us":178,"threads_started":1}
I20260811 15:11:01.175592 19405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:01.152434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:35045 (local address 127.30.8.62:43447)
0811 15:11:01.155694 (+ 3260us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:01.155721 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:01.155774 (+ 53us) server_negotiation.cc:407] Connection header received
0811 15:11:01.156127 (+ 353us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:01.156142 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:01.156608 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:01.157013 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:01.159071 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:01.160798 (+ 1727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:01.162592 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:01.163404 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:01.163584 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:01.166822 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:01.166829 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:01.166850 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:01.166888 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:01.170312 (+ 3424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:01.171164 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:01.171172 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:01.171178 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:01.171354 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:01.171857 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:01.171860 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:01.171862 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:01.172407 (+ 545us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:01.172539 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:01.174231 (+ 1692us) server_negotiation.cc:299] Negotiation successful
0811 15:11:01.174587 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3066,"thread_start_us":121,"threads_started":1}
I20260811 15:11:01.178433 19398 heartbeater.cc:347] Connected to a master server at 127.30.8.62:43447
I20260811 15:11:01.178704 19398 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:01.179355 19398 heartbeater.cc:511] Master 127.30.8.62:43447 requested a full tablet report, sending...
W20260811 15:11:01.181875 19398 heartbeater.cc:644] Failed to heartbeat to 127.30.8.62:43447 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 15:11:01.182015 19398 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:01.182219 19398 heartbeater.cc:511] Master 127.30.8.62:43447 requested a full tablet report, sending...
I20260811 15:11:01.183373 19398 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:01.183584 19398 heartbeater.cc:511] Master 127.30.8.62:43447 requested a full tablet report, sending...
W20260811 15:11:01.184474 19398 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 15:11:01.306231 19401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:01.306559 19401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:01.306622 19401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:01.312263 19401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:01.312369 19401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:01.312461 19401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:11:01.320595 19401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.8.20:57211/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:42983
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=34547
--tserver_master_addrs=127.30.8.62:43447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:01.322203 19401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:01.323755 19401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:01.336874 19413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:01.336879 19412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:01.337584 19416 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:01.337842 19401 server_base.cc:1034] running on GCE node
I20260811 15:11:01.338384 19401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:01.339241 19401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:01.340415 19401 hybrid_clock.cc:648] HybridClock initialized: now 1786461061340376 us; error 61 us; skew 500 ppm
Aug 11 15:11:01 dist-test-slave-1vmg krb5kdc[17362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:01.343581 19401 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:11:01.344640 19401 webserver.cc:460] Webserver started at http://127.30.8.2:34547/ using document root <none> and password file <none>
I20260811 15:11:01.345252 19401 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:01.345342 19401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 15:11:01 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
I20260811 15:11:01.370973 19401 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 15:11:01.374231 19431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:01.375881 19401 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 15:11:01.376013 19401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "8986648b7c4b42fa8f10976589b80188"
format_stamp: "Formatted at 2026-08-11 15:11:00 on dist-test-slave-1vmg"
server_key: "5b002fc6bec84bf6de08215489ce40f6"
server_key_iv: "694fc47667bda8b233d65c32271e0757"
server_key_version: "kuduclusterkey@0"
I20260811 15:11:01.376624 19401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:01.402343 19401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:01.406378 19401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:01.406690 19401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:01.407595 19401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:11:01.408834 19401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:11:01.408946 19401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:01.409055 19401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:11:01.409123 19401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:01.422277 19401 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:42983
I20260811 15:11:01.422644 19636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:42983 every 8 connection(s)
I20260811 15:11:01.423841 19401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:11:01.432382 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 19401
I20260811 15:11:01.432917 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.8.20:57211/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:38777
--local_ip_for_outbound_sockets=127.30.8.3
--tserver_master_addrs=127.30.8.62:43447
--webserver_port=35047
--webserver_interface=127.30.8.3
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:36077
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.8.20:57211/kms with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 15:11:01 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:01.441138 19405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:01.425862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:33183 (local address 127.30.8.62:43447)
0811 15:11:01.426044 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:01.426049 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:01.427004 (+ 955us) server_negotiation.cc:407] Connection header received
0811 15:11:01.428015 (+ 1011us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:01.428019 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:01.428081 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:01.428159 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:01.429923 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:01.430810 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:01.433019 (+ 2209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:01.433392 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:01.433475 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:01.436705 (+ 3230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:01.436710 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:01.436713 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:01.436748 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:01.438419 (+ 1671us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:01.439129 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:01.439133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:01.439135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:01.439232 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:01.439687 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:01.439691 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:01.439693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:01.439961 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:01.440046 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:01.440814 (+ 768us) server_negotiation.cc:299] Negotiation successful
0811 15:11:01.440961 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 15:11:01.443207 19639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:01.426299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:43447 (local address 127.30.8.2:33183)
0811 15:11:01.426814 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:01.426870 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:01.427795 (+ 925us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:01.428494 (+ 699us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:01.428503 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:01.428916 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:01.429735 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:01.429758 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:01.431210 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:01.431214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:01.432869 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:01.432884 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:01.433731 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:01.433735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:01.433950 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:01.434624 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:01.434646 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:01.436461 (+ 1815us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:01.438574 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:01.438581 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:01.438601 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:01.439003 (+ 402us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:01.439351 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:01.439354 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:01.439357 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:01.439565 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:01.440118 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:01.440126 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:01.440491 (+ 365us) client_negotiation.cc:769] Sending connection context
0811 15:11:01.441639 (+ 1148us) client_negotiation.cc:240] Negotiation successful
0811 15:11:01.442028 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":142,"threads_started":1}
I20260811 15:11:01.444983 19637 heartbeater.cc:347] Connected to a master server at 127.30.8.62:43447
I20260811 15:11:01.445266 19637 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:01.445920 19637 heartbeater.cc:511] Master 127.30.8.62:43447 requested a full tablet report, sending...
W20260811 15:11:01.447851 19637 heartbeater.cc:644] Failed to heartbeat to 127.30.8.62:43447 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 15:11:01.447990 19637 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:01.448203 19637 heartbeater.cc:511] Master 127.30.8.62:43447 requested a full tablet report, sending...
I20260811 15:11:01.449095 19637 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:01.449280 19637 heartbeater.cc:511] Master 127.30.8.62:43447 requested a full tablet report, sending...
W20260811 15:11:01.450023 19637 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 15:11:01.593334 19640 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:01.593756 19640 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:01.593827 19640 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:01.598146 19640 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:01.598228 19640 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:01.598315 19640 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:11:01.604452 19640 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36077
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.8.20:57211/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:38777
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=35047
--tserver_master_addrs=127.30.8.62:43447
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19640
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:01.605609 19640 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:01.606738 19640 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:01.619234 19651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:01.619491 19649 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:01.619596 19648 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:01.620572 19640 server_base.cc:1034] running on GCE node
I20260811 15:11:01.622830 19640 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:01.623919 19640 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:01.625181 19640 hybrid_clock.cc:648] HybridClock initialized: now 1786461061625147 us; error 56 us; skew 500 ppm
Aug 11 15:11:01 dist-test-slave-1vmg krb5kdc[17362](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:01.629159 19640 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:11:01.630663 19640 webserver.cc:460] Webserver started at http://127.30.8.3:35047/ using document root <none> and password file <none>
I20260811 15:11:01.631517 19640 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:01.631651 19640 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 15:11:01 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
I20260811 15:11:01.655344 19640 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 15:11:01.658869 19662 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:01.661772 19640 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.000s
I20260811 15:11:01.661942 19640 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "57d39dea94fc4c03bf550aade1632d07"
format_stamp: "Formatted at 2026-08-11 15:11:00 on dist-test-slave-1vmg"
server_key: "e863a5fb5f10c995dabc7a4bd1eeefc9"
server_key_iv: "6f54d39092425f54aace792c2e61f42a"
server_key_version: "kuduclusterkey@0"
I20260811 15:11:01.662868 19640 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:01.698792 19640 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:01.702836 19640 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:01.703208 19640 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:01.704125 19640 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:11:01.705152 19640 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:11:01.705201 19640 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:01.705246 19640 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:11:01.705276 19640 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:01.716643 19640 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:38777
I20260811 15:11:01.718181 19640 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:11:01.724237 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 19640
I20260811 15:11:01.735788 19873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:38777 every 8 connection(s)
I20260811 15:11:01.741674 19405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:01.725444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41626 (local address 127.30.8.62:43447)
0811 15:11:01.725614 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:01.725619 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:01.733002 (+ 7383us) server_negotiation.cc:407] Connection header received
0811 15:11:01.733165 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:01.733169 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:01.733238 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:01.733349 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:01.734397 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:01.735038 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:01.736153 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:01.736394 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:01.736486 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:01.738261 (+ 1775us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:01.738264 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:01.738267 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:01.738288 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:01.739984 (+ 1696us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:01.740378 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:01.740382 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:01.740384 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:01.740467 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:01.740722 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:01.740725 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:01.740727 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:01.740939 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:01.741160 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:01.741374 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 15:11:01.741501 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Aug 11 15:11:01 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461061, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:01.748857 19881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:01.735320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:42937 (local address 127.30.8.62:43447)
0811 15:11:01.735621 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:01.735631 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:01.737061 (+ 1430us) server_negotiation.cc:407] Connection header received
0811 15:11:01.737969 (+ 908us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:01.737972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:01.738028 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:01.738117 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:01.739906 (+ 1789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:01.740760 (+ 854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:01.742438 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:01.742723 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:01.742819 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:01.745288 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:01.745292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:01.745294 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:01.745319 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:01.746637 (+ 1318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:01.747227 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:01.747230 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:01.747232 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:01.747313 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:01.747656 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:01.747660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:01.747661 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:01.747857 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:01.747944 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:01.748529 (+ 585us) server_negotiation.cc:299] Negotiation successful
0811 15:11:01.748670 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":114,"threads_started":1}
I20260811 15:11:01.749993 19883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:01.735477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:43447 (local address 127.30.8.3:42937)
0811 15:11:01.736926 (+ 1449us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:01.736970 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:01.737775 (+ 805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:01.738623 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:01.738631 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:01.738992 (+ 361us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:01.739710 (+ 718us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:01.739750 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:01.740896 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:01.740900 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:01.742229 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:01.742241 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:01.742881 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:01.742886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:01.743125 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:01.743719 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:01.743736 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:01.745122 (+ 1386us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:01.746747 (+ 1625us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:01.746754 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:01.746773 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:01.747133 (+ 360us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:01.747394 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:01.747398 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:01.747400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:01.747552 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:01.747958 (+ 406us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:01.747966 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:01.748273 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 15:11:01.748456 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 15:11:01.748822 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1219,"thread_start_us":187,"threads_started":1}
I20260811 15:11:01.751598 19878 heartbeater.cc:347] Connected to a master server at 127.30.8.62:43447
I20260811 15:11:01.751863 19878 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:01.752471 19878 heartbeater.cc:511] Master 127.30.8.62:43447 requested a full tablet report, sending...
W20260811 15:11:01.754106 19878 heartbeater.cc:644] Failed to heartbeat to 127.30.8.62:43447 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 15:11:01.754248 19878 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:01.754434 19878 heartbeater.cc:511] Master 127.30.8.62:43447 requested a full tablet report, sending...
I20260811 15:11:01.755251 19878 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:01.755424 19878 heartbeater.cc:511] Master 127.30.8.62:43447 requested a full tablet report, sending...
W20260811 15:11:01.756244 19878 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 15:11:02 dist-test-slave-1vmg krb5kdc[17362](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786461062, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:02.184762 19398 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:02.185146 19398 heartbeater.cc:511] Master 127.30.8.62:43447 requested a full tablet report, sending...
I20260811 15:11:02.186568 19070 ts_manager.cc:194] Registered new tserver with Master: 3b9df95c364c4460a5a97e9045211b52 (127.30.8.1:36817)
I20260811 15:11:02.188297 19070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:35045
I20260811 15:11:02.450282 19637 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:02.450618 19637 heartbeater.cc:511] Master 127.30.8.62:43447 requested a full tablet report, sending...
I20260811 15:11:02.451550 19070 ts_manager.cc:194] Registered new tserver with Master: 8986648b7c4b42fa8f10976589b80188 (127.30.8.2:42983)
I20260811 15:11:02.452380 19070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:33183
Aug 11 15:11:02 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461062, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
I20260811 15:11:02.756488 19878 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:02.756829 19878 heartbeater.cc:511] Master 127.30.8.62:43447 requested a full tablet report, sending...
I20260811 15:11:02.757639 19070 ts_manager.cc:194] Registered new tserver with Master: 57d39dea94fc4c03bf550aade1632d07 (127.30.8.3:38777)
I20260811 15:11:02.758415 19070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:42937
I20260811 15:11:02.768018 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 15:11:02.782709 19911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:02.769900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41632 (local address 127.30.8.62:43447)
0811 15:11:02.770940 (+ 1040us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:02.770945 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:02.771537 (+ 592us) server_negotiation.cc:407] Connection header received
0811 15:11:02.771696 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:02.771700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:02.771757 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:02.771833 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:02.773245 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:02.774112 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:02.775312 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:02.776574 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:02.776699 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:02.778636 (+ 1937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:02.778641 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:02.778644 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:02.778679 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:02.780620 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:02.781160 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:02.781164 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:02.781166 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:02.781270 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:02.781670 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:02.781673 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:02.781675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:02.781936 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:02.782068 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:02.782389 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 15:11:02.782519 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":918,"thread_start_us":119,"threads_started":1}
Aug 11 15:11:02 dist-test-slave-1vmg krb5kdc[17362](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461062, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for HTTP/127.30.8.20@KRBTEST.COM
I20260811 15:11:02.787271 19070 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41632:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:11:02.792029 19070 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:11:02.839252 19928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:02.821718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:38777 (local address 127.0.0.1:51478)
0811 15:11:02.826660 (+ 4942us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:02.826675 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:02.826775 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:02.830539 (+ 3764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:02.830543 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:02.830563 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:02.830671 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:02.830678 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:02.836016 (+ 5338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:02.836020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:02.837169 (+ 1149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:02.837181 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:02.838591 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:02.838595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:02.838726 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:02.838774 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 15:11:02.838895 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 15:11:02.839078 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4871,"thread_start_us":90,"threads_started":1}
I20260811 15:11:02.839810 19922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:02.821006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33514 (local address 127.30.8.2:42983)
0811 15:11:02.831167 (+ 10161us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:02.831175 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:02.831203 (+ 28us) server_negotiation.cc:407] Connection header received
0811 15:11:02.831282 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:02.831288 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:02.831508 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:02.831664 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:02.832150 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:02.833211 (+ 1061us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:02.834792 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:02.835674 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:02.835782 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:02.839473 (+ 3691us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:02.839533 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 15:11:02.839645 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10010,"thread_start_us":91,"threads_started":1}
I20260811 15:11:02.839282 19921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:02.820870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:42983 (local address 127.0.0.1:33514)
0811 15:11:02.823662 (+ 2792us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:02.823723 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:02.823864 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:02.831872 (+ 8008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:02.831876 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:02.831897 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:02.831996 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:02.832003 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:02.833343 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:02.833347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:02.834634 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:02.834647 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:02.838944 (+ 4297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:02.838948 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:02.839030 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:02.839057 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 15:11:02.839124 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 15:11:02.839184 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2589,"mutex_wait_us":1,"thread_start_us":118,"threads_started":1}
I20260811 15:11:02.840835 19924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:02.822072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:36817 (local address 127.0.0.1:54416)
0811 15:11:02.826978 (+ 4906us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:02.826991 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:02.827083 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:02.827949 (+ 866us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:02.827957 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:02.827981 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:02.828149 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:02.828156 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:02.832512 (+ 4356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:02.832516 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:02.834861 (+ 2345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:02.834874 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:02.840448 (+ 5574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:02.840452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:02.840552 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:02.840578 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 15:11:02.840624 (+ 46us) client_negotiation.cc:240] Negotiation successful
0811 15:11:02.840698 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4822,"mutex_wait_us":1,"thread_start_us":83,"threads_started":1}
I20260811 15:11:02.841545 19929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:02.822264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54416 (local address 127.30.8.1:36817)
0811 15:11:02.826472 (+ 4208us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:02.826481 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:02.827184 (+ 703us) server_negotiation.cc:407] Connection header received
0811 15:11:02.827288 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:02.827295 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:02.827563 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:02.827726 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:02.829722 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:02.832358 (+ 2636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:02.835553 (+ 3195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:02.840295 (+ 4742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:02.841064 (+ 769us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:02.841227 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:02.841287 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 15:11:02.841391 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4069,"thread_start_us":136,"threads_started":1}
I20260811 15:11:02.842984 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:02.821881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51478 (local address 127.30.8.3:38777)
0811 15:11:02.828443 (+ 6562us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:02.828451 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:02.828476 (+ 25us) server_negotiation.cc:407] Connection header received
0811 15:11:02.828551 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:02.828556 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:02.830352 (+ 1796us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:02.831393 (+ 1041us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:02.831589 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:02.835875 (+ 4286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:02.837331 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:02.838211 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:02.838325 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:02.838505 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:02.842688 (+ 4183us) server_negotiation.cc:299] Negotiation successful
0811 15:11:02.842819 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6440,"thread_start_us":98,"threads_started":1}
I20260811 15:11:02.844652 19513 tablet_service.cc:1467] Processing CreateTablet for tablet bb728994b0ea4f7fbf3d685ebbdfd9b9 (DEFAULT_TABLE table=test-table [id=ae7070f11303485c88c29ec4fb8209ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:11:02.844656 19290 tablet_service.cc:1467] Processing CreateTablet for tablet bb728994b0ea4f7fbf3d685ebbdfd9b9 (DEFAULT_TABLE table=test-table [id=ae7070f11303485c88c29ec4fb8209ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:11:02.845880 19290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb728994b0ea4f7fbf3d685ebbdfd9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:02.845759 19732 tablet_service.cc:1467] Processing CreateTablet for tablet bb728994b0ea4f7fbf3d685ebbdfd9b9 (DEFAULT_TABLE table=test-table [id=ae7070f11303485c88c29ec4fb8209ca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:11:02.846776 19732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb728994b0ea4f7fbf3d685ebbdfd9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:02.848641 19513 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb728994b0ea4f7fbf3d685ebbdfd9b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:02.858348 19932 tablet_bootstrap.cc:492] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 3b9df95c364c4460a5a97e9045211b52: Bootstrap starting.
I20260811 15:11:02.858384 19933 tablet_bootstrap.cc:492] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188: Bootstrap starting.
I20260811 15:11:02.861025 19932 tablet_bootstrap.cc:654] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 3b9df95c364c4460a5a97e9045211b52: Neither blocks nor log segments found. Creating new log.
I20260811 15:11:02.861063 19933 tablet_bootstrap.cc:654] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188: Neither blocks nor log segments found. Creating new log.
I20260811 15:11:02.862025 19933 log.cc:824] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:02.863270 19932 log.cc:824] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 3b9df95c364c4460a5a97e9045211b52: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:02.864609 19933 tablet_bootstrap.cc:492] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188: No bootstrap required, opened a new log
I20260811 15:11:02.864830 19933 ts_tablet_manager.cc:1397] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 15:11:02.865489 19932 tablet_bootstrap.cc:492] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 3b9df95c364c4460a5a97e9045211b52: No bootstrap required, opened a new log
I20260811 15:11:02.865679 19932 ts_tablet_manager.cc:1397] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 3b9df95c364c4460a5a97e9045211b52: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.000s
I20260811 15:11:02.870167 19931 tablet_bootstrap.cc:492] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07: Bootstrap starting.
I20260811 15:11:02.870405 19932 raft_consensus.cc:348] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 3b9df95c364c4460a5a97e9045211b52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8986648b7c4b42fa8f10976589b80188" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42983 } } peers { permanent_uuid: "57d39dea94fc4c03bf550aade1632d07" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 38777 } } peers { permanent_uuid: "3b9df95c364c4460a5a97e9045211b52" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36817 } }
I20260811 15:11:02.870931 19932 raft_consensus.cc:374] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 3b9df95c364c4460a5a97e9045211b52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:02.871062 19932 raft_consensus.cc:729] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 3b9df95c364c4460a5a97e9045211b52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3b9df95c364c4460a5a97e9045211b52, State: Initialized, Role: FOLLOWER
I20260811 15:11:02.871735 19932 consensus_queue.cc:260] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 3b9df95c364c4460a5a97e9045211b52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8986648b7c4b42fa8f10976589b80188" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42983 } } peers { permanent_uuid: "57d39dea94fc4c03bf550aade1632d07" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 38777 } } peers { permanent_uuid: "3b9df95c364c4460a5a97e9045211b52" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36817 } }
I20260811 15:11:02.873234 19933 raft_consensus.cc:348] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8986648b7c4b42fa8f10976589b80188" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42983 } } peers { permanent_uuid: "57d39dea94fc4c03bf550aade1632d07" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 38777 } } peers { permanent_uuid: "3b9df95c364c4460a5a97e9045211b52" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36817 } }
I20260811 15:11:02.874276 19932 ts_tablet_manager.cc:1428] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 3b9df95c364c4460a5a97e9045211b52: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260811 15:11:02.874558 19933 raft_consensus.cc:374] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:02.874718 19933 raft_consensus.cc:729] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8986648b7c4b42fa8f10976589b80188, State: Initialized, Role: FOLLOWER
I20260811 15:11:02.874749 19398 heartbeater.cc:503] Master 127.30.8.62:43447 was elected leader, sending a full tablet report...
I20260811 15:11:02.875816 19933 consensus_queue.cc:260] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8986648b7c4b42fa8f10976589b80188" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42983 } } peers { permanent_uuid: "57d39dea94fc4c03bf550aade1632d07" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 38777 } } peers { permanent_uuid: "3b9df95c364c4460a5a97e9045211b52" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36817 } }
I20260811 15:11:02.877453 19931 tablet_bootstrap.cc:654] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07: Neither blocks nor log segments found. Creating new log.
I20260811 15:11:02.878434 19933 ts_tablet_manager.cc:1428] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188: Time spent starting tablet: real 0.014s user 0.007s sys 0.000s
I20260811 15:11:02.879820 19931 log.cc:824] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:02.879951 19637 heartbeater.cc:503] Master 127.30.8.62:43447 was elected leader, sending a full tablet report...
I20260811 15:11:02.886696 19931 tablet_bootstrap.cc:492] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07: No bootstrap required, opened a new log
I20260811 15:11:02.887081 19931 ts_tablet_manager.cc:1397] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07: Time spent bootstrapping tablet: real 0.017s user 0.007s sys 0.000s
W20260811 15:11:02.901510 19400 tablet.cc:2369] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 3b9df95c364c4460a5a97e9045211b52: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:11:02.891641 19931 raft_consensus.cc:348] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8986648b7c4b42fa8f10976589b80188" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42983 } } peers { permanent_uuid: "57d39dea94fc4c03bf550aade1632d07" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 38777 } } peers { permanent_uuid: "3b9df95c364c4460a5a97e9045211b52" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36817 } }
I20260811 15:11:02.904026 19931 raft_consensus.cc:374] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:02.904199 19931 raft_consensus.cc:729] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57d39dea94fc4c03bf550aade1632d07, State: Initialized, Role: FOLLOWER
I20260811 15:11:02.905015 19931 consensus_queue.cc:260] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8986648b7c4b42fa8f10976589b80188" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42983 } } peers { permanent_uuid: "57d39dea94fc4c03bf550aade1632d07" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 38777 } } peers { permanent_uuid: "3b9df95c364c4460a5a97e9045211b52" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36817 } }
I20260811 15:11:02.906493 19931 ts_tablet_manager.cc:1428] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07: Time spent starting tablet: real 0.019s user 0.006s sys 0.000s
I20260811 15:11:02.907094 19878 heartbeater.cc:503] Master 127.30.8.62:43447 was elected leader, sending a full tablet report...
W20260811 15:11:02.939345 19638 tablet.cc:2369] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 15:11:02.983417 19879 tablet.cc:2369] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:11:03.003623 19940 raft_consensus.cc:482] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:11:03.003881 19940 raft_consensus.cc:504] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8986648b7c4b42fa8f10976589b80188" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42983 } } peers { permanent_uuid: "57d39dea94fc4c03bf550aade1632d07" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 38777 } } peers { permanent_uuid: "3b9df95c364c4460a5a97e9045211b52" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36817 } }
I20260811 15:11:03.005383 19940 leader_election.cc:288] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57d39dea94fc4c03bf550aade1632d07 (127.30.8.3:38777), 3b9df95c364c4460a5a97e9045211b52 (127.30.8.1:36817)
I20260811 15:11:03.017463 19929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:03.006481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:53383 (local address 127.30.8.1:36817)
0811 15:11:03.006633 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:03.006638 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:03.012988 (+ 6350us) server_negotiation.cc:407] Connection header received
0811 15:11:03.013218 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:03.013221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:03.013280 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:03.013330 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:03.013759 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:03.014554 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:03.016126 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:03.016712 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:03.016815 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:03.016889 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:03.017232 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 15:11:03.017318 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 15:11:03.018160 19945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:03.006313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:36817 (local address 127.30.8.2:53383)
0811 15:11:03.012882 (+ 6569us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:03.012898 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:03.013074 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:03.013478 (+ 404us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:03.013481 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:03.013501 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:03.013625 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:03.013630 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:03.014668 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:03.014671 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:03.015972 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:03.015985 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:03.016956 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:03.016960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:03.017082 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:03.017110 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 15:11:03.017897 (+ 787us) client_negotiation.cc:240] Negotiation successful
0811 15:11:03.017985 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6488,"thread_start_us":110,"threads_started":1}
I20260811 15:11:03.018788 19927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:03.006828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:39091 (local address 127.30.8.3:38777)
0811 15:11:03.006974 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:03.006979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:03.010695 (+ 3716us) server_negotiation.cc:407] Connection header received
0811 15:11:03.011214 (+ 519us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:03.011219 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:03.011277 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:03.011327 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:03.014242 (+ 2915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:03.014992 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:03.017402 (+ 2410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:03.018045 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:03.018146 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:03.018213 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:03.018576 (+ 363us) server_negotiation.cc:299] Negotiation successful
0811 15:11:03.018667 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 15:11:03.019212 19944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:03.005871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:38777 (local address 127.30.8.2:39091)
0811 15:11:03.010558 (+ 4687us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:03.010579 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:03.011028 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:03.013200 (+ 2172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:03.013203 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:03.013226 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:03.013377 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:03.013383 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:03.016052 (+ 2669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:03.016055 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:03.017240 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:03.017252 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:03.018322 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:03.018326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:03.018432 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:03.018454 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 15:11:03.018973 (+ 519us) client_negotiation.cc:240] Negotiation successful
0811 15:11:03.019055 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4587,"thread_start_us":174,"threads_started":1}
I20260811 15:11:03.019366 19311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb728994b0ea4f7fbf3d685ebbdfd9b9" candidate_uuid: "8986648b7c4b42fa8f10976589b80188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b9df95c364c4460a5a97e9045211b52" is_pre_election: true
I20260811 15:11:03.019780 19311 raft_consensus.cc:2454] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 3b9df95c364c4460a5a97e9045211b52 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8986648b7c4b42fa8f10976589b80188 in term 0.
I20260811 15:11:03.020637 19780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb728994b0ea4f7fbf3d685ebbdfd9b9" candidate_uuid: "8986648b7c4b42fa8f10976589b80188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57d39dea94fc4c03bf550aade1632d07" is_pre_election: true
I20260811 15:11:03.020926 19780 raft_consensus.cc:2454] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8986648b7c4b42fa8f10976589b80188 in term 0.
I20260811 15:11:03.021569 19432 leader_election.cc:302] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b9df95c364c4460a5a97e9045211b52, 8986648b7c4b42fa8f10976589b80188; no voters:
I20260811 15:11:03.022178 19940 raft_consensus.cc:2781] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:11:03.022276 19940 raft_consensus.cc:482] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:11:03.022317 19940 raft_consensus.cc:3037] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:03.023777 19940 raft_consensus.cc:504] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8986648b7c4b42fa8f10976589b80188" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42983 } } peers { permanent_uuid: "57d39dea94fc4c03bf550aade1632d07" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 38777 } } peers { permanent_uuid: "3b9df95c364c4460a5a97e9045211b52" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36817 } }
I20260811 15:11:03.024216 19940 leader_election.cc:288] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [CANDIDATE]: Term 1 election: Requested vote from peers 57d39dea94fc4c03bf550aade1632d07 (127.30.8.3:38777), 3b9df95c364c4460a5a97e9045211b52 (127.30.8.1:36817)
I20260811 15:11:03.025279 19780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb728994b0ea4f7fbf3d685ebbdfd9b9" candidate_uuid: "8986648b7c4b42fa8f10976589b80188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57d39dea94fc4c03bf550aade1632d07"
I20260811 15:11:03.025414 19780 raft_consensus.cc:3037] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:03.026804 19780 raft_consensus.cc:2454] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8986648b7c4b42fa8f10976589b80188 in term 1.
I20260811 15:11:03.027199 19311 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb728994b0ea4f7fbf3d685ebbdfd9b9" candidate_uuid: "8986648b7c4b42fa8f10976589b80188" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3b9df95c364c4460a5a97e9045211b52"
I20260811 15:11:03.027338 19311 raft_consensus.cc:3037] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 3b9df95c364c4460a5a97e9045211b52 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:03.028661 19311 raft_consensus.cc:2454] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 3b9df95c364c4460a5a97e9045211b52 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8986648b7c4b42fa8f10976589b80188 in term 1.
I20260811 15:11:03.029218 19432 leader_election.cc:302] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3b9df95c364c4460a5a97e9045211b52, 8986648b7c4b42fa8f10976589b80188; no voters:
I20260811 15:11:03.029474 19940 raft_consensus.cc:2781] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:11:03.029726 19940 raft_consensus.cc:686] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [term 1 LEADER]: Becoming Leader. State: Replica: 8986648b7c4b42fa8f10976589b80188, State: Running, Role: LEADER
I20260811 15:11:03.030184 19940 consensus_queue.cc:237] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8986648b7c4b42fa8f10976589b80188" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42983 } } peers { permanent_uuid: "57d39dea94fc4c03bf550aade1632d07" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 38777 } } peers { permanent_uuid: "3b9df95c364c4460a5a97e9045211b52" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36817 } }
I20260811 15:11:03.034685 19070 catalog_manager.cc:5495] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8986648b7c4b42fa8f10976589b80188 (127.30.8.2). New cstate: current_term: 1 leader_uuid: "8986648b7c4b42fa8f10976589b80188" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8986648b7c4b42fa8f10976589b80188" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42983 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "57d39dea94fc4c03bf550aade1632d07" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 38777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3b9df95c364c4460a5a97e9045211b52" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36817 } health_report { overall_health: UNKNOWN } } }
I20260811 15:11:03.070318 19922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:03.064618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33526 (local address 127.30.8.2:42983)
0811 15:11:03.064784 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:03.064790 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:03.064810 (+ 20us) server_negotiation.cc:407] Connection header received
0811 15:11:03.064912 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:03.064915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:03.065002 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:03.065173 (+ 171us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:11:03.065447 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:03.066249 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:03.067468 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:03.067850 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:03.067946 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:03.068516 (+ 570us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:11:03.069196 (+ 680us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:11:03.069311 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:03.070070 (+ 759us) server_negotiation.cc:299] Negotiation successful
0811 15:11:03.070145 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 15:11:03.078969 19311 raft_consensus.cc:1261] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 3b9df95c364c4460a5a97e9045211b52 [term 1 FOLLOWER]: Refusing update from remote peer 8986648b7c4b42fa8f10976589b80188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:11:03.079963 19948 consensus_queue.cc:1035] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3b9df95c364c4460a5a97e9045211b52" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 36817 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:11:03.088634 19780 raft_consensus.cc:1261] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07 [term 1 FOLLOWER]: Refusing update from remote peer 8986648b7c4b42fa8f10976589b80188: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:11:03.089675 19948 consensus_queue.cc:1035] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [LEADER]: Connected to new peer: Peer: permanent_uuid: "57d39dea94fc4c03bf550aade1632d07" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 38777 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:11:03.102721 19956 mvcc.cc:205] Tried to move back new op lower bound from 7317344514359889920 to 7317344514177216512. Current Snapshot: MvccSnapshot[applied={T|T < 7317344514359889920}]
I20260811 15:11:03.105363 19952 mvcc.cc:205] Tried to move back new op lower bound from 7317344514359889920 to 7317344514177216512. Current Snapshot: MvccSnapshot[applied={T|T < 7317344514359889920}]
I20260811 15:11:03.105702 19954 mvcc.cc:205] Tried to move back new op lower bound from 7317344514359889920 to 7317344514177216512. Current Snapshot: MvccSnapshot[applied={T|T < 7317344514359889920}]
I20260811 15:11:03.117651 19070 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:11:03.117956 19070 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41632:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:11:03.120708 19070 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff319b1a69c24cd397beded0cfbaae75: Sending DeleteTablet for 3 replicas of tablet bb728994b0ea4f7fbf3d685ebbdfd9b9
I20260811 15:11:03.122552 19513 tablet_service.cc:1514] Processing DeleteTablet for tablet bb728994b0ea4f7fbf3d685ebbdfd9b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:11:03 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:33514
I20260811 15:11:03.123185 19732 tablet_service.cc:1514] Processing DeleteTablet for tablet bb728994b0ea4f7fbf3d685ebbdfd9b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:11:03 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:51478
I20260811 15:11:03.123709 19290 tablet_service.cc:1514] Processing DeleteTablet for tablet bb728994b0ea4f7fbf3d685ebbdfd9b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:11:03 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:54416
I20260811 15:11:03.125362 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 19136
I20260811 15:11:03.130054 19969 tablet_replica.cc:330] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188: stopping tablet replica
I20260811 15:11:03.130545 19969 raft_consensus.cc:2229] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:11:03.131076 19969 raft_consensus.cc:2258] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:11:03.131580 19970 tablet_replica.cc:330] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07: stopping tablet replica
I20260811 15:11:03.131978 19970 raft_consensus.cc:2229] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:11:03.136325 19969 ts_tablet_manager.cc:1905] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:11:03.139443 19969 ts_tablet_manager.cc:1918] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:11:03.132344 19970 raft_consensus.cc:2258] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:11:03.140245 19969 log.cc:1196] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/bb728994b0ea4f7fbf3d685ebbdfd9b9
I20260811 15:11:03.140645 19969 ts_tablet_manager.cc:1939] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 8986648b7c4b42fa8f10976589b80188: Deleting consensus metadata
I20260811 15:11:03.141824 19970 ts_tablet_manager.cc:1905] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:11:03.144604 19970 ts_tablet_manager.cc:1918] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:11:03.146289 19042 catalog_manager.cc:4841] TS 8986648b7c4b42fa8f10976589b80188 (127.30.8.2:42983): tablet bb728994b0ea4f7fbf3d685ebbdfd9b9 (table test-table [id=ae7070f11303485c88c29ec4fb8209ca]) successfully deleted
I20260811 15:11:03.146699 19970 log.cc:1196] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/bb728994b0ea4f7fbf3d685ebbdfd9b9
I20260811 15:11:03.147177 19970 ts_tablet_manager.cc:1939] T bb728994b0ea4f7fbf3d685ebbdfd9b9 P 57d39dea94fc4c03bf550aade1632d07: Deleting consensus metadata
W20260811 15:11:03.148269 19042 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.8.1:36817 (error 108)
I20260811 15:11:03.148940 19042 catalog_manager.cc:4841] TS 57d39dea94fc4c03bf550aade1632d07 (127.30.8.3:38777): tablet bb728994b0ea4f7fbf3d685ebbdfd9b9 (table test-table [id=ae7070f11303485c88c29ec4fb8209ca]) successfully deleted
I20260811 15:11:03.149618 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 19401
I20260811 15:11:03.149648 19924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:03.149306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:36817 (local address 127.0.0.1:54428)
0811 15:11:03.149482 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:03.149559 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.1:36817: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99}
W20260811 15:11:03.149796 19042 catalog_manager.cc:4584] TS 3b9df95c364c4460a5a97e9045211b52 (127.30.8.1:36817): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet bb728994b0ea4f7fbf3d685ebbdfd9b9: Network error: Client connection negotiation failed: client connection to 127.30.8.1:36817: connect: Connection refused (error 111)
I20260811 15:11:03.183455 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 19640
I20260811 15:11:03.200680 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 19018
I20260811 15:11:03.217286 30752 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 15:11:03.563292 30752 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 15:11:03.563454 30752 mini_ranger.cc:66] Stopping Ranger...
I20260811 15:11:03.909169 30752 mini_ranger.cc:68] Stopped Ranger
2026-08-11 15:11:03.910 UTC [17400] LOG: received smart shutdown request
2026-08-11 15:11:03.914 UTC [17400] LOG: background worker "logical replication launcher" (PID 17412) exited with exit code 1
2026-08-11 15:11:03.915 UTC [17405] LOG: shutting down
2026-08-11 15:11:04.054 UTC [17400] LOG: database system is shut down
2026-08-11T15:11:04Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (52300 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:11:04 dist-test-slave-1vmg krb5kdc[19992](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:11:04 dist-test-slave-1vmg krb5kdc[19992](info): set up 2 sockets
Aug 11 15:11:04 dist-test-slave-1vmg krb5kdc[19992](info): commencing operation
krb5kdc: starting...
W20260811 15:11:05.335294 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.152s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:11:05 dist-test-slave-1vmg krb5kdc[19992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:11:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:11:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:11:05.467983 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:42269
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:35593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:42269,127.30.8.61:38205,127.30.8.60:37433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:11:05.588740 20014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:05.589071 20014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:05.589130 20014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:05.593308 20014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:11:05.593393 20014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:05.593420 20014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:05.593442 20014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:11:05.593463 20014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:11:05.599151 20014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:35593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:42269,127.30.8.61:38205,127.30.8.60:37433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:42269
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:05.600394 20014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:05.601459 20014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:05.610273 20027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:05.610273 20024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:05.610273 20021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:05.610332 20014 server_base.cc:1034] running on GCE node
I20260811 15:11:05.611152 20014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:05.612335 20014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:05.613504 20014 hybrid_clock.cc:648] HybridClock initialized: now 1786461065613491 us; error 37 us; skew 500 ppm
Aug 11 15:11:05 dist-test-slave-1vmg krb5kdc[19992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:05.616421 20014 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:11:05.617393 20014 webserver.cc:460] Webserver started at http://127.30.8.62:35379/ using document root <none> and password file <none>
I20260811 15:11:05.617981 20014 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:05.618052 20014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:05.618263 20014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:11:05.620494 20014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "4fc42a07cdea4d2e958ed97faca14b44"
format_stamp: "Formatted at 2026-08-11 15:11:05 on dist-test-slave-1vmg"
server_key: "9b20fbf6b87e301a906657c367d325f5"
server_key_iv: "3686693cd2d8bd7989bd7de4c8719fbd"
server_key_version: "encryptionkey@0"
I20260811 15:11:05.621073 20014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "4fc42a07cdea4d2e958ed97faca14b44"
format_stamp: "Formatted at 2026-08-11 15:11:05 on dist-test-slave-1vmg"
server_key: "9b20fbf6b87e301a906657c367d325f5"
server_key_iv: "3686693cd2d8bd7989bd7de4c8719fbd"
server_key_version: "encryptionkey@0"
I20260811 15:11:05.625048 20014 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:11:05.627611 20042 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:05.628871 20014 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:11:05.629004 20014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "4fc42a07cdea4d2e958ed97faca14b44"
format_stamp: "Formatted at 2026-08-11 15:11:05 on dist-test-slave-1vmg"
server_key: "9b20fbf6b87e301a906657c367d325f5"
server_key_iv: "3686693cd2d8bd7989bd7de4c8719fbd"
server_key_version: "encryptionkey@0"
I20260811 15:11:05.629118 20014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:05.642061 20014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:05.646010 20014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:05.646250 20014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:05.653422 20014 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:42269
I20260811 15:11:05.653458 20122 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:42269 every 8 connection(s)
I20260811 15:11:05.654459 20014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:11:05.655841 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 20014
I20260811 15:11:05.655978 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:11:05.656296 30752 external_mini_cluster.cc:1428] Setting key b10ad1dc92541a30ba4c7de94df90fdf
I20260811 15:11:05.658033 20124 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:05.662226 20124 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.8.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.61@KRBTEST.COM" created.
Aug 11 15:11:05 dist-test-slave-1vmg krb5kdc[19992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:05.678247 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.664652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51964 (local address 127.30.8.62:42269)
0811 15:11:05.665150 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:05.665166 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:05.665355 (+ 189us) server_negotiation.cc:407] Connection header received
0811 15:11:05.666450 (+ 1095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:05.666455 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:05.666686 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:05.666880 (+ 194us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:05.667763 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:05.668643 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:05.669991 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:05.670552 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:05.670729 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:05.673153 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:05.673162 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:05.673180 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:05.673210 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:05.675075 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:05.675549 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:05.675556 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:05.675558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:05.675706 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:05.675986 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:05.675989 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:05.675991 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:05.676606 (+ 615us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:05.676757 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:05.677110 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 15:11:05.677436 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":88,"server-negotiator.queue_time_us":365,"spinlock_wait_cycles":32000,"thread_start_us":185,"threads_started":1}
I20260811 15:11:05.678244 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.664651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:42269 (local address 127.0.0.1:51964)
0811 15:11:05.665189 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:05.665240 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:05.666160 (+ 920us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:05.666883 (+ 723us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:05.666888 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:05.667291 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:05.667596 (+ 305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:05.667622 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:05.668779 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:05.668783 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:05.669862 (+ 1079us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:05.669873 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:05.670740 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:05.670743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:05.670841 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:05.671482 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:05.671499 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:05.672989 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:05.675184 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:05.675193 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:05.675203 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:05.675435 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:05.675793 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:05.675795 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:05.675797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:05.675902 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:05.676709 (+ 807us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:05.676714 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:05.676843 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 15:11:05.676990 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 15:11:05.677366 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"mutex_wait_us":61,"thread_start_us":134,"threads_started":1}
I20260811 15:11:05.680883 20124 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } has no permanent_uuid. Determining permanent_uuid...
I20260811 15:11:05.681631 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.681368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:38205 (local address 127.0.0.1:56508)
0811 15:11:05.681489 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:05.681564 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.61:38205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 15:11:05.681849 20047 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.8.61:38205: connect: Connection refused (error 111)
I20260811 15:11:05.682753 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.682554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:38205 (local address 127.0.0.1:56512)
0811 15:11:05.682654 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:05.682698 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.61:38205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 15:11:05.682888 20124 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.61:38205: Network error: Client connection negotiation failed: client connection to 127.30.8.61:38205: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.61.keytab.
Entry for principal kudu/127.30.8.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.61.keytab.
I20260811 15:11:05.685581 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.61:38205
--webserver_interface=127.30.8.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.61.keytab
--principal=kudu/127.30.8.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:35593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:42269,127.30.8.61:38205,127.30.8.60:37433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:11:05.732183 20124 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } attempt: 1
I20260811 15:11:05.733158 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.732849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:38205 (local address 127.0.0.1:56526)
0811 15:11:05.733013 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:05.733076 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.61:38205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 15:11:05.733986 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.733748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:38205 (local address 127.0.0.1:56542)
0811 15:11:05.733865 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:05.733913 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.61:38205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 15:11:05.734107 20124 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.61:38205: Network error: Client connection negotiation failed: client connection to 127.30.8.61:38205: connect: Connection refused (error 111)
I20260811 15:11:05.802253 20124 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } attempt: 2
I20260811 15:11:05.803136 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.802802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:38205 (local address 127.0.0.1:56544)
0811 15:11:05.802985 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:05.803046 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.61:38205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 15:11:05.803900 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.803695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:38205 (local address 127.0.0.1:56552)
0811 15:11:05.803795 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:05.803841 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.61:38205: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 15:11:05.804011 20124 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.61:38205: Network error: Client connection negotiation failed: client connection to 127.30.8.61:38205: connect: Connection refused (error 111)
W20260811 15:11:05.805814 20137 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:05.806099 20137 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:05.806149 20137 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:05.810303 20137 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:11:05.810371 20137 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:05.810390 20137 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:05.810406 20137 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:11:05.810420 20137 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:11:05.816151 20137 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:35593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:42269,127.30.8.61:38205,127.30.8.60:37433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.61.keytab
--principal=kudu/127.30.8.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.61:38205
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.8.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20137
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:05.817327 20137 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:05.818344 20137 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:05.827008 20149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:05.827009 20154 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:05.827009 20152 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:05.827111 20137 server_base.cc:1034] running on GCE node
I20260811 15:11:05.827852 20137 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:05.829059 20137 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:05.830230 20137 hybrid_clock.cc:648] HybridClock initialized: now 1786461065830209 us; error 32 us; skew 500 ppm
Aug 11 15:11:05 dist-test-slave-1vmg krb5kdc[19992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:05.833182 20137 init.cc:377] Logged in from keytab as kudu/127.30.8.61@KRBTEST.COM (short username kudu)
I20260811 15:11:05.834170 20137 webserver.cc:460] Webserver started at http://127.30.8.61:33839/ using document root <none> and password file <none>
I20260811 15:11:05.834849 20137 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:05.834962 20137 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:05.835198 20137 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:11:05.837368 20137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-1/data/instance:
uuid: "0bc9f0dd338348f1a1c40b75489de49b"
format_stamp: "Formatted at 2026-08-11 15:11:05 on dist-test-slave-1vmg"
server_key: "c219add105589cdcb53c6debb0d9a80d"
server_key_iv: "2e0543e7d93da52860f9dfcea0b6befd"
server_key_version: "encryptionkey@0"
I20260811 15:11:05.837949 20137 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-1/wal/instance:
uuid: "0bc9f0dd338348f1a1c40b75489de49b"
format_stamp: "Formatted at 2026-08-11 15:11:05 on dist-test-slave-1vmg"
server_key: "c219add105589cdcb53c6debb0d9a80d"
server_key_iv: "2e0543e7d93da52860f9dfcea0b6befd"
server_key_version: "encryptionkey@0"
I20260811 15:11:05.842398 20137 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 15:11:05.844988 20167 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:05.846230 20137 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 15:11:05.846369 20137 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-1/wal
uuid: "0bc9f0dd338348f1a1c40b75489de49b"
format_stamp: "Formatted at 2026-08-11 15:11:05 on dist-test-slave-1vmg"
server_key: "c219add105589cdcb53c6debb0d9a80d"
server_key_iv: "2e0543e7d93da52860f9dfcea0b6befd"
server_key_version: "encryptionkey@0"
I20260811 15:11:05.846477 20137 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:05.881762 20137 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:05.885013 20137 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:05.885195 20137 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:05.892769 20137 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.61:38205
I20260811 15:11:05.892793 20274 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.61:38205 every 8 connection(s)
I20260811 15:11:05.893784 20137 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-1/data/info.pb
I20260811 15:11:05.895147 20124 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } attempt: 3
I20260811 15:11:05.897862 20275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:05.902613 20275 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 15:11:05 dist-test-slave-1vmg krb5kdc[19992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for kudu/127.30.8.61@KRBTEST.COM
I20260811 15:11:05.903817 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 20137
I20260811 15:11:05.903947 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-1/wal/instance
I20260811 15:11:05.904222 30752 external_mini_cluster.cc:1428] Setting key e83387fb2f72b6f69f1647c19af38227
I20260811 15:11:05.908612 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.895677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:38205 (local address 127.0.0.1:56558)
0811 15:11:05.895858 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:05.895876 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:05.895985 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:05.898187 (+ 2202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:05.898190 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:05.898401 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:05.898554 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:05.898563 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:05.899916 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:05.899920 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:05.900740 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:05.900755 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:05.901445 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:05.901449 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:05.901561 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:05.902100 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:05.902117 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:05.903573 (+ 1456us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:05.906643 (+ 3070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:05.906647 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:05.906649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:05.906863 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:05.907240 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:05.907244 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:05.907246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:05.907353 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:05.908067 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:05.908070 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:05.908180 (+ 110us) client_negotiation.cc:769] Sending connection context
0811 15:11:05.908298 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 15:11:05.908441 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 15:11:05.909765 20279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.896406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56558 (local address 127.30.8.61:38205)
0811 15:11:05.896809 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:05.896824 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:05.896860 (+ 36us) server_negotiation.cc:407] Connection header received
0811 15:11:05.897504 (+ 644us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:05.897520 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:05.897884 (+ 364us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:05.898195 (+ 311us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:05.898688 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:05.899777 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:05.900857 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:05.901320 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:05.901441 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:05.903826 (+ 2385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:05.903833 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:05.903850 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:05.903885 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:05.906535 (+ 2650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:05.907008 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:05.907013 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:05.907018 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:05.907141 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:05.907441 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:05.907445 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:05.907446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:05.907953 (+ 507us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:05.908088 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:05.908467 (+ 379us) server_negotiation.cc:299] Negotiation successful
0811 15:11:05.908803 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":261,"thread_start_us":116,"threads_started":1}
Aug 11 15:11:05 dist-test-slave-1vmg krb5kdc[19992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.61@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:05.911995 20124 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } has no permanent_uuid. Determining permanent_uuid...
I20260811 15:11:05.912814 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.912520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37433 (local address 127.0.0.1:34506)
0811 15:11:05.912671 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:05.912731 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260811 15:11:05.913574 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.913305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37433 (local address 127.0.0.1:34512)
0811 15:11:05.913463 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:05.913513 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
W20260811 15:11:05.913774 20124 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.60:37433: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37433: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.30.8.60@KRBTEST.COM; defaulting to no policy
I20260811 15:11:05.915498 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.904723 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51966 (local address 127.30.8.62:42269)
0811 15:11:05.904864 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:05.904868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:05.905153 (+ 285us) server_negotiation.cc:407] Connection header received
0811 15:11:05.905367 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:05.905371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:05.905424 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:05.905476 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:05.906245 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:05.907048 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:05.908162 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:05.909064 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:05.909169 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:05.911906 (+ 2737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:05.911910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:05.911913 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:05.911940 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:05.913573 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:05.914176 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:05.914180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:05.914182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:05.914262 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:05.914517 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:05.914520 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:05.914522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:05.914723 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:05.914810 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:05.915224 (+ 414us) server_negotiation.cc:299] Negotiation successful
0811 15:11:05.915360 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 15:11:05.915531 20282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.904632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:42269 (local address 127.0.0.1:51966)
0811 15:11:05.905028 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:05.905078 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:05.905257 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:05.905649 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:05.905654 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:05.906000 (+ 346us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:05.906123 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:05.906129 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:05.907161 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:05.907165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:05.908040 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:05.908049 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:05.909188 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:05.909192 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:05.909326 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:05.910262 (+ 936us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:05.910278 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:05.911728 (+ 1450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:05.913819 (+ 2091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:05.913829 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:05.913835 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:05.914078 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:05.914348 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:05.914352 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:05.914353 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:05.914437 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:05.914826 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:05.914833 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:05.915002 (+ 169us) client_negotiation.cc:769] Sending connection context
0811 15:11:05.915136 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 15:11:05.915362 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":116,"threads_started":1}
I20260811 15:11:05.916882 20275 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.60@KRBTEST.COM" created.
Aug 11 15:11:05 dist-test-slave-1vmg krb5kdc[19992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.61@KRBTEST.COM for kudu/127.30.8.61@KRBTEST.COM
I20260811 15:11:05.924643 20282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.917408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:38205 (local address 127.0.0.1:56570)
0811 15:11:05.917515 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:05.917529 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:05.917606 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:05.917860 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:05.917863 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:05.918016 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:05.918104 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:05.918109 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:05.918836 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:05.918839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:05.919386 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:05.919395 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:05.919840 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:05.919844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:05.919932 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:05.920415 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:05.920427 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:05.921616 (+ 1189us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:05.923261 (+ 1645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:05.923264 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:05.923267 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:05.923450 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:05.923686 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:05.923689 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:05.923691 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:05.923767 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:05.924115 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:05.924118 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:05.924205 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 15:11:05.924298 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 15:11:05.924450 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260811 15:11:05.924643 20279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.917532 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56570 (local address 127.30.8.61:38205)
0811 15:11:05.917649 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:05.917654 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:05.917668 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:11:05.917705 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:05.917708 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:05.917764 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:05.917846 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:05.918209 (+ 363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:05.918734 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:05.919492 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:05.919743 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:05.919832 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:05.921749 (+ 1917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:05.921752 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:05.921754 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:05.921775 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:05.923155 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:05.923535 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:05.923538 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:05.923539 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:05.923608 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:05.923842 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:05.923845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:05.923847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:05.924029 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:05.924114 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:05.924355 (+ 241us) server_negotiation.cc:299] Negotiation successful
0811 15:11:05.924505 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":41}
I20260811 15:11:05.925510 20275 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } has no permanent_uuid. Determining permanent_uuid...
I20260811 15:11:05.926290 20282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.926017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37433 (local address 127.0.0.1:34522)
0811 15:11:05.926138 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:05.926218 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 15:11:05.926791 20169 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37433: connect: Connection refused (error 111)
I20260811 15:11:05.927693 20282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.927512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37433 (local address 127.0.0.1:34538)
0811 15:11:05.927597 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:05.927639 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260811 15:11:05.927825 20275 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.60:37433: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37433: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.60.keytab.
Entry for principal kudu/127.30.8.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.60.keytab.
I20260811 15:11:05.933262 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.60:37433
--webserver_interface=127.30.8.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.60.keytab
--principal=kudu/127.30.8.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:35593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:42269,127.30.8.61:38205,127.30.8.60:37433
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:11:05.944938 20124 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } attempt: 1
I20260811 15:11:05.945808 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.945504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37433 (local address 127.0.0.1:34544)
0811 15:11:05.945656 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:05.945716 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 15:11:05.946553 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.946363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37433 (local address 127.0.0.1:34560)
0811 15:11:05.946454 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:05.946497 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 15:11:05.946671 20124 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.60:37433: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37433: connect: Connection refused (error 111)
I20260811 15:11:05.977085 20275 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } attempt: 1
I20260811 15:11:05.977970 20282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.977666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37433 (local address 127.0.0.1:34574)
0811 15:11:05.977830 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:05.977891 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 15:11:05.978822 20282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:05.978597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37433 (local address 127.0.0.1:34578)
0811 15:11:05.978705 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:05.978762 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 15:11:05.978973 20275 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.60:37433: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37433: connect: Connection refused (error 111)
I20260811 15:11:06.021835 20124 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } attempt: 2
I20260811 15:11:06.022677 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:06.022389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37433 (local address 127.0.0.1:34582)
0811 15:11:06.022537 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:06.022594 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 15:11:06.023538 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:06.023298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37433 (local address 127.0.0.1:34594)
0811 15:11:06.023418 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:06.023472 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 15:11:06.023710 20124 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.60:37433: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37433: connect: Connection refused (error 111)
I20260811 15:11:06.047148 20275 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } attempt: 2
I20260811 15:11:06.048028 20282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:06.047736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37433 (local address 127.0.0.1:34606)
0811 15:11:06.047890 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:06.047952 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 15:11:06.048837 20282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:06.048617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37433 (local address 127.0.0.1:34618)
0811 15:11:06.048732 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:06.048780 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37433: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 15:11:06.048979 20275 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.60:37433: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37433: connect: Connection refused (error 111)
W20260811 15:11:06.053397 20286 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:06.053704 20286 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:06.053781 20286 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:06.057979 20286 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:11:06.058060 20286 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:06.058089 20286 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:06.058112 20286 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:11:06.058141 20286 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:11:06.063943 20286 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:35593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:42269,127.30.8.61:38205,127.30.8.60:37433
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.60.keytab
--principal=kudu/127.30.8.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.60:37433
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.8.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20286
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:06.065163 20286 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:06.066258 20286 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:06.074965 20294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:06.074967 20298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:06.074967 20291 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:06.075168 20286 server_base.cc:1034] running on GCE node
I20260811 15:11:06.075976 20286 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:06.077179 20286 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:06.078354 20286 hybrid_clock.cc:648] HybridClock initialized: now 1786461066078337 us; error 32 us; skew 500 ppm
Aug 11 15:11:06 dist-test-slave-1vmg krb5kdc[19992](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:06.081180 20286 init.cc:377] Logged in from keytab as kudu/127.30.8.60@KRBTEST.COM (short username kudu)
I20260811 15:11:06.082194 20286 webserver.cc:460] Webserver started at http://127.30.8.60:38995/ using document root <none> and password file <none>
I20260811 15:11:06.082787 20286 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:06.082856 20286 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:06.083110 20286 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:11:06.085377 20286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-2/data/instance:
uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c"
format_stamp: "Formatted at 2026-08-11 15:11:06 on dist-test-slave-1vmg"
server_key: "787ec875c577b9143c26b2a716770ed2"
server_key_iv: "228139ab6114240ff9d7bac3034b7904"
server_key_version: "encryptionkey@0"
I20260811 15:11:06.085939 20286 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-2/wal/instance:
uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c"
format_stamp: "Formatted at 2026-08-11 15:11:06 on dist-test-slave-1vmg"
server_key: "787ec875c577b9143c26b2a716770ed2"
server_key_iv: "228139ab6114240ff9d7bac3034b7904"
server_key_version: "encryptionkey@0"
I20260811 15:11:06.089841 20286 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 15:11:06.092428 20310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:06.093742 20286 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 15:11:06.093883 20286 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-2/wal
uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c"
format_stamp: "Formatted at 2026-08-11 15:11:06 on dist-test-slave-1vmg"
server_key: "787ec875c577b9143c26b2a716770ed2"
server_key_iv: "228139ab6114240ff9d7bac3034b7904"
server_key_version: "encryptionkey@0"
I20260811 15:11:06.094000 20286 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:06.106091 20286 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:06.109153 20286 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:06.109366 20286 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:06.116843 20286 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.60:37433
I20260811 15:11:06.116856 20402 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.60:37433 every 8 connection(s)
I20260811 15:11:06.117880 20286 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-2/data/info.pb
I20260811 15:11:06.120945 20403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:06.121002 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 20286
I20260811 15:11:06.121116 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786460893564915-30752-0/minicluster-data/master-2/wal/instance
I20260811 15:11:06.121425 30752 external_mini_cluster.cc:1428] Setting key 5254e25fef5d933e160c988d3c5d24f8
I20260811 15:11:06.122874 20124 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } attempt: 3
I20260811 15:11:06.126799 20403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 15:11:06 dist-test-slave-1vmg krb5kdc[19992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for kudu/127.30.8.60@KRBTEST.COM
Aug 11 15:11:06 dist-test-slave-1vmg krb5kdc[19992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.60@KRBTEST.COM
Aug 11 15:11:06 dist-test-slave-1vmg krb5kdc[19992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.60@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:06.135916 20127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:06.123337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37433 (local address 127.0.0.1:34648)
0811 15:11:06.123525 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:06.123541 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:06.123682 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:06.124960 (+ 1278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:06.124964 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:06.125187 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:06.125336 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:06.125344 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.126973 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.126977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:06.127801 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:06.127813 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.128674 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.128678 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:06.128772 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:06.129505 (+ 733us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:06.129522 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:06.130865 (+ 1343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:06.133685 (+ 2820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:06.133688 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:06.133690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:06.133966 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:06.134277 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:06.134281 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:06.134283 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:06.134369 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:06.135295 (+ 926us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:06.135299 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:06.135397 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 15:11:06.135495 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 15:11:06.135642 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 15:11:06.137152 20406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:06.122816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34634 (local address 127.30.8.60:37433)
0811 15:11:06.123291 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:06.123312 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:06.123355 (+ 43us) server_negotiation.cc:407] Connection header received
0811 15:11:06.124260 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:06.124281 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:06.124668 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:06.125001 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:06.125958 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.126747 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.127969 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.128551 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.128891 (+ 340us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:06.131608 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:06.131611 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:06.131614 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:06.131639 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:06.133488 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.134026 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.134032 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.134038 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.134176 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.134521 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.134524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.134526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.135098 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:06.135288 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:06.135605 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 15:11:06.135938 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1066,"server-negotiator.queue_time_us":328,"spinlock_wait_cycles":14464,"thread_start_us":160,"threads_started":1}
I20260811 15:11:06.137152 20407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:06.123578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34648 (local address 127.30.8.60:37433)
0811 15:11:06.123877 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:06.123884 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:06.123895 (+ 11us) server_negotiation.cc:407] Connection header received
0811 15:11:06.124264 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:06.124285 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:06.124672 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:06.125002 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:06.125537 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.126782 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.127937 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.128546 (+ 609us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.128739 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:06.131120 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:06.131127 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:06.131140 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:06.131173 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:06.133524 (+ 2351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.134060 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.134064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.134066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.134180 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.134451 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.134454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.134456 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.135098 (+ 642us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:06.135244 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:06.136138 (+ 894us) server_negotiation.cc:299] Negotiation successful
0811 15:11:06.136262 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":219,"server-negotiator.queue_time_us":242,"spinlock_wait_cycles":31360,"thread_start_us":127,"threads_started":1}
I20260811 15:11:06.140015 20409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:06.128865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:42269 (local address 127.0.0.1:51974)
0811 15:11:06.129287 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:06.129345 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:06.129462 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:06.130251 (+ 789us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:06.130257 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:06.130690 (+ 433us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:06.130831 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:06.130837 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.131792 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.131795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:06.132705 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:06.132718 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.133314 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.133318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:06.133440 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:06.133909 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:06.133924 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:06.136175 (+ 2251us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:06.137932 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:06.137938 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:06.137942 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:06.138227 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:06.138552 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:06.138555 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:06.138557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:06.138656 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:06.139350 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:06.139356 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:06.139484 (+ 128us) client_negotiation.cc:769] Sending connection context
0811 15:11:06.139637 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 15:11:06.139848 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"spinlock_wait_cycles":9472,"thread_start_us":131,"threads_started":1}
I20260811 15:11:06.140298 20275 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } attempt: 3
I20260811 15:11:06.140581 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:06.129748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51974 (local address 127.30.8.62:42269)
0811 15:11:06.129892 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:06.129896 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:06.129910 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:11:06.129949 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:06.129952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:06.129999 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:06.130080 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:06.130973 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.131662 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.132837 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.133175 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.133300 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:06.136305 (+ 3005us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:06.136309 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:06.136312 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:06.136337 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:06.137832 (+ 1495us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.138333 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.138337 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.138338 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.138445 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.138775 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.138778 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.138780 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.139129 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:06.139236 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:06.139710 (+ 474us) server_negotiation.cc:299] Negotiation successful
0811 15:11:06.140412 (+ 702us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 15:11:06.141960 20124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44: Bootstrap starting.
I20260811 15:11:06.144058 20403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } has no permanent_uuid. Determining permanent_uuid...
I20260811 15:11:06.144759 20124 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44: Neither blocks nor log segments found. Creating new log.
Aug 11 15:11:06 dist-test-slave-1vmg krb5kdc[19992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.61@KRBTEST.COM for kudu/127.30.8.60@KRBTEST.COM
I20260811 15:11:06.145737 20124 log.cc:824] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:06.148880 20282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:06.140737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37433 (local address 127.0.0.1:34654)
0811 15:11:06.140857 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:06.140872 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:06.140957 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:06.141175 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:06.141177 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:06.141323 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:06.141431 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:06.141439 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.142185 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.142189 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:06.142710 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:06.142719 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.143337 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.143341 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:06.143424 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:06.143963 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:06.143981 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:06.145476 (+ 1495us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:06.147250 (+ 1774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:06.147254 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:06.147256 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:06.147506 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:06.147804 (+ 298us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:06.147807 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:06.147809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:06.147902 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:06.148349 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:06.148353 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:06.148457 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 15:11:06.148555 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 15:11:06.148695 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 15:11:06.149358 20406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:06.140788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34654 (local address 127.30.8.60:37433)
0811 15:11:06.140890 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:06.140894 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:06.140912 (+ 18us) server_negotiation.cc:407] Connection header received
0811 15:11:06.141039 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:06.141042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:06.141087 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:06.141166 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:06.141546 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.142086 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.142845 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.143186 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.144576 (+ 1390us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:06.145631 (+ 1055us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:06.145635 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:06.145638 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:06.145661 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:06.147140 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.147590 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.147594 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.147596 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.147683 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.147998 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.148001 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.148002 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.148206 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:06.148302 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:06.149100 (+ 798us) server_negotiation.cc:299] Negotiation successful
0811 15:11:06.149221 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 15:11:06.149725 20124 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44: No bootstrap required, opened a new log
Aug 11 15:11:06 dist-test-slave-1vmg krb5kdc[19992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.60@KRBTEST.COM for kudu/127.30.8.61@KRBTEST.COM
Aug 11 15:11:06 dist-test-slave-1vmg krb5kdc[19992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:06.153528 20275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b: Bootstrap starting.
I20260811 15:11:06.153774 20124 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc42a07cdea4d2e958ed97faca14b44" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } } peers { permanent_uuid: "0bc9f0dd338348f1a1c40b75489de49b" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } } peers { permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } }
I20260811 15:11:06.154026 20124 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:06.154078 20124 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fc42a07cdea4d2e958ed97faca14b44, State: Initialized, Role: FOLLOWER
I20260811 15:11:06.154625 20124 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc42a07cdea4d2e958ed97faca14b44" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } } peers { permanent_uuid: "0bc9f0dd338348f1a1c40b75489de49b" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } } peers { permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } }
I20260811 15:11:06.154987 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:06.143043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51984 (local address 127.30.8.62:42269)
0811 15:11:06.143173 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:06.143177 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:06.143191 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:11:06.143225 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:06.143228 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:06.143446 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:06.143519 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:06.145268 (+ 1749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.146038 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.147175 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.147509 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.147579 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:06.151204 (+ 3625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:06.151209 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:06.151212 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:06.151241 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:06.152837 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.153316 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.153319 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.153321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.153414 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.153745 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.153748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.153750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.154152 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:06.154343 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:06.154554 (+ 211us) server_negotiation.cc:299] Negotiation successful
0811 15:11:06.154678 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 15:11:06.156131 20124 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:11:06.156471 20275 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b: Neither blocks nor log segments found. Creating new log.
I20260811 15:11:06.157439 20275 log.cc:824] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:06.159523 20409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:06.144530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:38205 (local address 127.0.0.1:56582)
0811 15:11:06.144913 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:06.144927 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:06.145023 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:06.145358 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:06.145361 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:06.145536 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:06.145650 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:06.145657 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.146949 (+ 1292us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.146952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:06.147720 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:06.147731 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.148350 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.148353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:06.148455 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:06.148834 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:06.148849 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:06.155317 (+ 6468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:06.157638 (+ 2321us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:06.157641 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:06.157643 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:06.157948 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:06.158300 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:06.158303 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:06.158304 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:06.158410 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:06.158966 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:06.158969 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:06.159087 (+ 118us) client_negotiation.cc:769] Sending connection context
0811 15:11:06.159196 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 15:11:06.159349 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205}
I20260811 15:11:06.160029 20275 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b: No bootstrap required, opened a new log
I20260811 15:11:06.160348 20279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:06.144768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56582 (local address 127.30.8.61:38205)
0811 15:11:06.144878 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:06.144882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:06.144998 (+ 116us) server_negotiation.cc:407] Connection header received
0811 15:11:06.145115 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:06.145117 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:06.145164 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:06.145246 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:06.145783 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.146786 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.147847 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.148132 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.148202 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:06.155488 (+ 7286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:06.155492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:06.155495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:06.155524 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:06.157224 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.158066 (+ 842us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.158070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.158072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.158173 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.158520 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.158523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.158525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.158799 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:06.158890 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:06.160082 (+ 1192us) server_negotiation.cc:299] Negotiation successful
0811 15:11:06.160206 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 15:11:06.160974 20403 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 15:11:06 dist-test-slave-1vmg krb5kdc[19992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461065, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.61@KRBTEST.COM
I20260811 15:11:06.164628 20275 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc42a07cdea4d2e958ed97faca14b44" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } } peers { permanent_uuid: "0bc9f0dd338348f1a1c40b75489de49b" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } } peers { permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } }
I20260811 15:11:06.164901 20275 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:06.164973 20275 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0bc9f0dd338348f1a1c40b75489de49b, State: Initialized, Role: FOLLOWER
I20260811 15:11:06.165339 20411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc42a07cdea4d2e958ed97faca14b44" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } } peers { permanent_uuid: "0bc9f0dd338348f1a1c40b75489de49b" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } } peers { permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } } }
I20260811 15:11:06.165498 20411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:11:06.166240 20275 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc42a07cdea4d2e958ed97faca14b44" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } } peers { permanent_uuid: "0bc9f0dd338348f1a1c40b75489de49b" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } } peers { permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } }
I20260811 15:11:06.162325 20014 master_runner.cc:186] Error getting master registration for 127.30.8.61:38205: OK, instance_id { permanent_uuid: "0bc9f0dd338348f1a1c40b75489de49b" instance_seqno: 1786461065890695 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 15:11:06.166482 20014 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
Aug 11 15:11:06 dist-test-slave-1vmg krb5kdc[19992](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.60@KRBTEST.COM for kudu/127.30.8.60@KRBTEST.COM
I20260811 15:11:06.168354 20275 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:11:06.169595 20415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:06.156682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56592 (local address 127.30.8.61:38205)
0811 15:11:06.158483 (+ 1801us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:06.158493 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:06.158505 (+ 12us) server_negotiation.cc:407] Connection header received
0811 15:11:06.158535 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:06.158538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:06.158580 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:06.159507 (+ 927us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:06.159615 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.160400 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.161378 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.161702 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.162159 (+ 457us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:06.164926 (+ 2767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:06.164931 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:06.164934 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:06.164960 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:06.167074 (+ 2114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.167590 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.167593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.167595 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.167673 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.168012 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.168015 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.168017 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.168249 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:06.169188 (+ 939us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:06.169268 (+ 80us) server_negotiation.cc:299] Negotiation successful
0811 15:11:06.169395 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1727,"thread_start_us":117,"threads_started":1}
W20260811 15:11:06.170341 20434 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
I20260811 15:11:06.170403 20409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:06.161552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37433 (local address 127.0.0.1:34660)
0811 15:11:06.161883 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:06.161895 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:06.161980 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:06.162280 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:06.162282 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:06.162454 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:06.162554 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:06.162562 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.163584 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.163588 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:06.164371 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:06.164384 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.165143 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.165147 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:06.165225 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:06.165718 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:06.165734 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:06.167193 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:06.168950 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:06.168952 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:06.168954 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:06.169177 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:06.169457 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:06.169459 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:06.169461 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:06.169529 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:06.169927 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:06.169930 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:06.170009 (+ 79us) client_negotiation.cc:769] Sending connection context
0811 15:11:06.170101 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 15:11:06.170251 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286}
W20260811 15:11:06.170421 20434 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:11:06.170893 20406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:06.161727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34660 (local address 127.30.8.60:37433)
0811 15:11:06.161824 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:06.161829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:06.161950 (+ 121us) server_negotiation.cc:407] Connection header received
0811 15:11:06.162056 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:06.162059 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:06.162106 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:06.162177 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:06.162672 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.163478 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.164546 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.164873 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.164959 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:06.167334 (+ 2375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:06.167338 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:06.167340 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:06.167364 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:06.168845 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.169292 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.169295 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.169297 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.169372 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.169612 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.169615 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.169617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.169816 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:06.169898 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:06.170651 (+ 753us) server_negotiation.cc:299] Negotiation successful
0811 15:11:06.170759 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 15:11:06.174609 20403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c: Bootstrap starting.
W20260811 15:11:06.174639 20455 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:11:06.174712 20455 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:11:06.174991 20435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc42a07cdea4d2e958ed97faca14b44" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } } peers { permanent_uuid: "0bc9f0dd338348f1a1c40b75489de49b" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } } peers { permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } } }
I20260811 15:11:06.175117 20435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:11:06.176805 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:11:06.177479 20403 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c: Neither blocks nor log segments found. Creating new log.
I20260811 15:11:06.178400 20403 log.cc:824] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:06.176256 20137 master_runner.cc:186] Error getting master registration for 127.30.8.60:37433: OK, instance_id { permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" instance_seqno: 1786461066114534 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 15:11:06.179558 20137 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 15:11:06.187676 20403 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c: No bootstrap required, opened a new log
I20260811 15:11:06.192277 20403 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc42a07cdea4d2e958ed97faca14b44" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } } peers { permanent_uuid: "0bc9f0dd338348f1a1c40b75489de49b" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } } peers { permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } }
I20260811 15:11:06.192574 20403 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:06.192627 20403 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec9f2e9ddd7848d5b04e82d5d0ab174c, State: Initialized, Role: FOLLOWER
I20260811 15:11:06.193176 20403 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc42a07cdea4d2e958ed97faca14b44" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } } peers { permanent_uuid: "0bc9f0dd338348f1a1c40b75489de49b" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } } peers { permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } }
I20260811 15:11:06.194768 20403 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:11:06.195465 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:06.180439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51986 (local address 127.30.8.62:42269)
0811 15:11:06.180553 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:06.180557 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:06.181326 (+ 769us) server_negotiation.cc:407] Connection header received
0811 15:11:06.181542 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:06.181547 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:06.181599 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:06.181655 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:06.182654 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.183475 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.184717 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.185045 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.185128 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:06.188854 (+ 3726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:06.188858 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:06.188862 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:06.188894 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:06.190936 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.192541 (+ 1605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.192545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.192547 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.192645 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.193625 (+ 980us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.193629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.193630 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.193890 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:06.194012 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:06.195126 (+ 1114us) server_negotiation.cc:299] Negotiation successful
0811 15:11:06.195288 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 15:11:06.197731 20415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:06.180883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56598 (local address 127.30.8.61:38205)
0811 15:11:06.181009 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:06.181013 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:06.181582 (+ 569us) server_negotiation.cc:407] Connection header received
0811 15:11:06.181762 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:06.181765 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:06.181814 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:06.181866 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:06.182798 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.183648 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.185710 (+ 2062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.186088 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.186200 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:06.189316 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:06.189320 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:06.189323 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:06.189355 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:06.191707 (+ 2352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.193104 (+ 1397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.193108 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.193110 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.193237 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.194345 (+ 1108us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.194349 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.194351 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.194578 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:06.194687 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:06.197422 (+ 2735us) server_negotiation.cc:299] Negotiation successful
0811 15:11:06.197567 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 15:11:06.199848 20470 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc42a07cdea4d2e958ed97faca14b44" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } } peers { permanent_uuid: "0bc9f0dd338348f1a1c40b75489de49b" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } } peers { permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } } }
I20260811 15:11:06.200014 20470 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:11:06.206035 20406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:06.180026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34674 (local address 127.30.8.60:37433)
0811 15:11:06.180167 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:06.180172 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:06.185616 (+ 5444us) server_negotiation.cc:407] Connection header received
0811 15:11:06.195191 (+ 9575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:06.195195 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:06.195260 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:06.195310 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:06.197255 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.198037 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.199345 (+ 1308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:06.199640 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:06.199762 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:06.202092 (+ 2330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:06.202096 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:06.202099 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:06.202128 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:06.204109 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.204612 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.204616 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.204618 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.204712 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:06.205017 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:06.205021 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:06.205023 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:06.205271 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:06.205375 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:06.205746 (+ 371us) server_negotiation.cc:299] Negotiation successful
0811 15:11:06.205873 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 15:11:06.207276 20286 master_runner.cc:263] No leader master found from master ec9f2e9ddd7848d5b04e82d5d0ab174c
I20260811 15:11:06.207346 20286 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 15:11:06.215454 20486 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:11:06.215577 20486 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:11:06.234565 20435 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:11:06.234802 20435 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc42a07cdea4d2e958ed97faca14b44" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } } peers { permanent_uuid: "0bc9f0dd338348f1a1c40b75489de49b" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } } peers { permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } }
I20260811 15:11:06.236092 20435 leader_election.cc:288] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 4fc42a07cdea4d2e958ed97faca14b44 (127.30.8.62:42269), ec9f2e9ddd7848d5b04e82d5d0ab174c (127.30.8.60:37433)
I20260811 15:11:06.236925 20087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0bc9f0dd338348f1a1c40b75489de49b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fc42a07cdea4d2e958ed97faca14b44" is_pre_election: true
I20260811 15:11:06.237283 20087 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bc9f0dd338348f1a1c40b75489de49b in term 0.
I20260811 15:11:06.238065 20357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0bc9f0dd338348f1a1c40b75489de49b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" is_pre_election: true
I20260811 15:11:06.238416 20357 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0bc9f0dd338348f1a1c40b75489de49b in term 0.
I20260811 15:11:06.238793 20170 leader_election.cc:302] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bc9f0dd338348f1a1c40b75489de49b, 4fc42a07cdea4d2e958ed97faca14b44; no voters:
I20260811 15:11:06.242343 20435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:11:06.242470 20435 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:11:06.242522 20435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:06.243952 20435 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc42a07cdea4d2e958ed97faca14b44" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } } peers { permanent_uuid: "0bc9f0dd338348f1a1c40b75489de49b" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } } peers { permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } }
I20260811 15:11:06.244724 20357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0bc9f0dd338348f1a1c40b75489de49b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c"
I20260811 15:11:06.244867 20357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:06.246258 20357 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bc9f0dd338348f1a1c40b75489de49b in term 1.
I20260811 15:11:06.246625 20169 leader_election.cc:302] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0bc9f0dd338348f1a1c40b75489de49b, ec9f2e9ddd7848d5b04e82d5d0ab174c; no voters:
I20260811 15:11:06.247357 20087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0bc9f0dd338348f1a1c40b75489de49b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4fc42a07cdea4d2e958ed97faca14b44"
I20260811 15:11:06.247510 20087 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:06.248893 20087 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0bc9f0dd338348f1a1c40b75489de49b in term 1.
I20260811 15:11:06.253252 20435 leader_election.cc:288] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [CANDIDATE]: Term 1 election: Requested vote from peers 4fc42a07cdea4d2e958ed97faca14b44 (127.30.8.62:42269), ec9f2e9ddd7848d5b04e82d5d0ab174c (127.30.8.60:37433)
I20260811 15:11:06.253463 20435 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:11:06.253559 20435 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [term 1 LEADER]: Becoming Leader. State: Replica: 0bc9f0dd338348f1a1c40b75489de49b, State: Running, Role: LEADER
I20260811 15:11:06.253980 20435 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc42a07cdea4d2e958ed97faca14b44" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } } peers { permanent_uuid: "0bc9f0dd338348f1a1c40b75489de49b" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } } peers { permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } }
I20260811 15:11:06.256162 20435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bc9f0dd338348f1a1c40b75489de49b. Latest consensus state: current_term: 1 leader_uuid: "0bc9f0dd338348f1a1c40b75489de49b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc42a07cdea4d2e958ed97faca14b44" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } } peers { permanent_uuid: "0bc9f0dd338348f1a1c40b75489de49b" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } } peers { permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } } }
I20260811 15:11:06.256302 20435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [sys.catalog]: This master's current role is: LEADER
I20260811 15:11:06.256872 20491 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:11:06.257923 20491 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:11:06.269675 20357 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c [term 1 FOLLOWER]: Refusing update from remote peer 0bc9f0dd338348f1a1c40b75489de49b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:11:06.270464 20488 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:11:06.277460 20470 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bc9f0dd338348f1a1c40b75489de49b. Latest consensus state: current_term: 1 leader_uuid: "0bc9f0dd338348f1a1c40b75489de49b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc42a07cdea4d2e958ed97faca14b44" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } } peers { permanent_uuid: "0bc9f0dd338348f1a1c40b75489de49b" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } } peers { permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } } }
I20260811 15:11:06.277631 20470 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:11:06.279678 20087 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44 [term 1 FOLLOWER]: Refusing update from remote peer 0bc9f0dd338348f1a1c40b75489de49b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:11:06.280421 20488 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4fc42a07cdea4d2e958ed97faca14b44" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:11:06.284814 20411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0bc9f0dd338348f1a1c40b75489de49b. Latest consensus state: current_term: 1 leader_uuid: "0bc9f0dd338348f1a1c40b75489de49b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc42a07cdea4d2e958ed97faca14b44" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } } peers { permanent_uuid: "0bc9f0dd338348f1a1c40b75489de49b" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } } peers { permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } } }
I20260811 15:11:06.284965 20411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:11:06.294396 20488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0bc9f0dd338348f1a1c40b75489de49b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc42a07cdea4d2e958ed97faca14b44" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } } peers { permanent_uuid: "0bc9f0dd338348f1a1c40b75489de49b" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } } peers { permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } } }
I20260811 15:11:06.294579 20488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [sys.catalog]: This master's current role is: LEADER
I20260811 15:11:06.295907 20411 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0bc9f0dd338348f1a1c40b75489de49b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc42a07cdea4d2e958ed97faca14b44" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } } peers { permanent_uuid: "0bc9f0dd338348f1a1c40b75489de49b" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } } peers { permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } } }
I20260811 15:11:06.296056 20411 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:11:06.296501 20495 mvcc.cc:205] Tried to move back new op lower bound from 7317344527431630848 to 7317344527380168704. Current Snapshot: MvccSnapshot[applied={T|T < 7317344527431630848}]
I20260811 15:11:06.299975 20435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0bc9f0dd338348f1a1c40b75489de49b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc42a07cdea4d2e958ed97faca14b44" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } } peers { permanent_uuid: "0bc9f0dd338348f1a1c40b75489de49b" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } } peers { permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } } }
I20260811 15:11:06.300124 20435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b [sys.catalog]: This master's current role is: LEADER
I20260811 15:11:06.301294 20470 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0bc9f0dd338348f1a1c40b75489de49b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4fc42a07cdea4d2e958ed97faca14b44" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42269 } } peers { permanent_uuid: "0bc9f0dd338348f1a1c40b75489de49b" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 38205 } } peers { permanent_uuid: "ec9f2e9ddd7848d5b04e82d5d0ab174c" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37433 } } }
I20260811 15:11:06.301434 20470 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:11:06.303833 20496 mvcc.cc:205] Tried to move back new op lower bound from 7317344527431630848 to 7317344527380168704. Current Snapshot: MvccSnapshot[applied={T|T < 7317344527431630848}]
I20260811 15:11:06.304422 20500 mvcc.cc:205] Tried to move back new op lower bound from 7317344527431630848 to 7317344527380168704. Current Snapshot: MvccSnapshot[applied={T|T < 7317344527431630848}]
I20260811 15:11:06.306519 20491 catalog_manager.cc:1347] Generated new cluster ID: 5237e239a6094fffa0166104da213f86
I20260811 15:11:06.306592 20491 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:11:06.326992 20491 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:11:06.331543 20491 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:11:06.355217 20491 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0bc9f0dd338348f1a1c40b75489de49b: Generated new TSK 0
I20260811 15:11:06.356297 20491 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:11:07.171865 20434 catalog_manager.cc:1259] Loaded cluster ID: 5237e239a6094fffa0166104da213f86
I20260811 15:11:07.171957 20434 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44: loading cluster ID for follower catalog manager: success
I20260811 15:11:07.173640 20434 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44: acquiring CA information for follower catalog manager: success
I20260811 15:11:07.174903 20434 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4fc42a07cdea4d2e958ed97faca14b44: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 15:11:07.217013 20486 catalog_manager.cc:1259] Loaded cluster ID: 5237e239a6094fffa0166104da213f86
I20260811 15:11:07.217106 20486 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c: loading cluster ID for follower catalog manager: success
I20260811 15:11:07.218869 20486 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c: acquiring CA information for follower catalog manager: success
I20260811 15:11:07.220158 20486 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ec9f2e9ddd7848d5b04e82d5d0ab174c: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 15:11:07.423806 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 20014
I20260811 15:11:07.440970 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 20137
I20260811 15:11:07.457527 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 20286
2026-08-11T15:11:07Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3306 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:11:07 dist-test-slave-1vmg krb5kdc[20526](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:11:07 dist-test-slave-1vmg krb5kdc[20526](info): set up 2 sockets
Aug 11 15:11:07 dist-test-slave-1vmg krb5kdc[20526](info): commencing operation
krb5kdc: starting...
W20260811 15:11:08.630519 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:11:08 dist-test-slave-1vmg krb5kdc[20526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:11:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:11:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:11:08.762463 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:34849
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:43339
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:34849
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:11:08.880904 20547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:08.881238 20547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:08.881300 20547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:08.885492 20547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:11:08.885586 20547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:08.885613 20547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:08.885633 20547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:11:08.885654 20547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:11:08.891328 20547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:43339
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:34849
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:34849
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:08.892558 20547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:08.893647 20547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:08.902395 20558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:08.902462 20562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:08.902531 20547 server_base.cc:1034] running on GCE node
W20260811 15:11:08.902395 20559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:08.903383 20547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:08.904598 20547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:08.905766 20547 hybrid_clock.cc:648] HybridClock initialized: now 1786461068905750 us; error 33 us; skew 500 ppm
Aug 11 15:11:08 dist-test-slave-1vmg krb5kdc[20526](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461068, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:08.908622 20547 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:11:08.909626 20547 webserver.cc:460] Webserver started at http://127.30.8.62:44301/ using document root <none> and password file <none>
I20260811 15:11:08.910183 20547 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:08.910252 20547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:08.910440 20547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:11:08.912607 20547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "1e6080ccc3864597b71d27478dfe813b"
format_stamp: "Formatted at 2026-08-11 15:11:08 on dist-test-slave-1vmg"
server_key: "cb8589a4d7d8271072f4c9560b9a942e"
server_key_iv: "f0598327efddee380d001aa720fe9618"
server_key_version: "encryptionkey@0"
I20260811 15:11:08.913149 20547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "1e6080ccc3864597b71d27478dfe813b"
format_stamp: "Formatted at 2026-08-11 15:11:08 on dist-test-slave-1vmg"
server_key: "cb8589a4d7d8271072f4c9560b9a942e"
server_key_iv: "f0598327efddee380d001aa720fe9618"
server_key_version: "encryptionkey@0"
I20260811 15:11:08.917078 20547 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:11:08.919637 20574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:08.920928 20547 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:11:08.921060 20547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "1e6080ccc3864597b71d27478dfe813b"
format_stamp: "Formatted at 2026-08-11 15:11:08 on dist-test-slave-1vmg"
server_key: "cb8589a4d7d8271072f4c9560b9a942e"
server_key_iv: "f0598327efddee380d001aa720fe9618"
server_key_version: "encryptionkey@0"
I20260811 15:11:08.921191 20547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:08.932894 20547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:08.936043 20547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:08.936247 20547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:08.943513 20547 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:34849
I20260811 15:11:08.943548 20662 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:34849 every 8 connection(s)
I20260811 15:11:08.944562 20547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:11:08.947535 20664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:08.949672 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 20547
I20260811 15:11:08.949807 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:11:08.950112 30752 external_mini_cluster.cc:1428] Setting key e1afa38efdf20d3a58dee37c21b0be04
I20260811 15:11:08.954355 20664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b: Bootstrap starting.
I20260811 15:11:08.957162 20664 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b: Neither blocks nor log segments found. Creating new log.
I20260811 15:11:08.958094 20664 log.cc:824] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b: Log is configured to *not* fsync() on all Append() calls
Aug 11 15:11:08 dist-test-slave-1vmg krb5kdc[20526](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461068, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:08.960260 20664 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b: No bootstrap required, opened a new log
I20260811 15:11:08.964998 20664 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e6080ccc3864597b71d27478dfe813b" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34849 } }
I20260811 15:11:08.965292 20664 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:08.965350 20664 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e6080ccc3864597b71d27478dfe813b, State: Initialized, Role: FOLLOWER
I20260811 15:11:08.965466 20667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:08.951709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46270 (local address 127.30.8.62:34849)
0811 15:11:08.952235 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:08.952256 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:08.952303 (+ 47us) server_negotiation.cc:407] Connection header received
0811 15:11:08.953106 (+ 803us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:08.953131 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:08.953575 (+ 444us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:08.953945 (+ 370us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:08.954638 (+ 693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:08.956128 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:08.956976 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:08.957681 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:08.957843 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:08.960326 (+ 2483us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:08.960333 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:08.960351 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:08.960383 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:08.962268 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:08.962691 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:08.962698 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:08.962706 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:08.962857 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:08.963129 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:08.963133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:08.963135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:08.963685 (+ 550us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:08.963902 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:08.964092 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 15:11:08.964435 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":127,"threads_started":1}
I20260811 15:11:08.965873 20664 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e6080ccc3864597b71d27478dfe813b" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34849 } }
I20260811 15:11:08.966018 20664 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:11:08.966085 20664 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:11:08.966184 20664 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:08.967213 20664 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e6080ccc3864597b71d27478dfe813b" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34849 } }
I20260811 15:11:08.967588 20664 leader_election.cc:302] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1e6080ccc3864597b71d27478dfe813b; no voters:
I20260811 15:11:08.967936 20664 leader_election.cc:288] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:11:08.968022 20670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:11:08.968254 20670 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b [term 1 LEADER]: Becoming Leader. State: Replica: 1e6080ccc3864597b71d27478dfe813b, State: Running, Role: LEADER
I20260811 15:11:08.968618 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:11:08.968643 20670 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e6080ccc3864597b71d27478dfe813b" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34849 } }
I20260811 15:11:08.969256 20664 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:11:08.969962 20670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1e6080ccc3864597b71d27478dfe813b. Latest consensus state: current_term: 1 leader_uuid: "1e6080ccc3864597b71d27478dfe813b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e6080ccc3864597b71d27478dfe813b" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34849 } } }
I20260811 15:11:08.970077 20670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b [sys.catalog]: This master's current role is: LEADER
I20260811 15:11:08.970384 20674 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1e6080ccc3864597b71d27478dfe813b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1e6080ccc3864597b71d27478dfe813b" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 34849 } } }
I20260811 15:11:08.970538 20674 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b [sys.catalog]: This master's current role is: LEADER
I20260811 15:11:08.971205 20684 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:11:08.974159 20684 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:11:08.980110 20684 catalog_manager.cc:1347] Generated new cluster ID: 3a2f51f8dec44a24bd9e8e35ac6348c2
I20260811 15:11:08.980206 20684 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:11:08.989995 20684 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:11:08.991107 20684 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:11:08.997629 20684 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1e6080ccc3864597b71d27478dfe813b: Generated new TSK 0
I20260811 15:11:08.998391 20684 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:11:09.025445 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 20547
2026-08-11T15:11:09Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1565 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:11:09 dist-test-slave-1vmg krb5kdc[20707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:11:09 dist-test-slave-1vmg krb5kdc[20707](info): set up 2 sockets
Aug 11 15:11:09 dist-test-slave-1vmg krb5kdc[20707](info): commencing operation
krb5kdc: starting...
W20260811 15:11:10.196426 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:11:10 dist-test-slave-1vmg krb5kdc[20707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:11:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:11:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:11:10.325917 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:45969
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:45969
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:11:10.443017 20729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:10.443310 20729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:10.443352 20729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:10.447500 20729 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:11:10.447583 20729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:10.447602 20729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:10.447625 20729 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:11:10.447641 20729 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:11:10.453243 20729 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:45969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:45969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:10.454447 20729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:10.455575 20729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:10.464313 20740 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:10.464314 20737 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:10.464313 20735 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:10.464358 20729 server_base.cc:1034] running on GCE node
I20260811 15:11:10.465183 20729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:10.466408 20729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:10.467583 20729 hybrid_clock.cc:648] HybridClock initialized: now 1786461070467553 us; error 46 us; skew 500 ppm
Aug 11 15:11:10 dist-test-slave-1vmg krb5kdc[20707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461070, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:10.470709 20729 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:11:10.471825 20729 webserver.cc:460] Webserver started at http://127.30.8.62:43883/ using document root <none> and password file <none>
I20260811 15:11:10.472429 20729 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:10.472499 20729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:10.472711 20729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:11:10.474951 20729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "50c523135e324a059998f8c92d341b97"
format_stamp: "Formatted at 2026-08-11 15:11:10 on dist-test-slave-1vmg"
server_key: "d6fd09097442b44787fb9a5d76c9096d"
server_key_iv: "af884792ead4d1ed38f96cd65cdd9297"
server_key_version: "encryptionkey@0"
I20260811 15:11:10.475546 20729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "50c523135e324a059998f8c92d341b97"
format_stamp: "Formatted at 2026-08-11 15:11:10 on dist-test-slave-1vmg"
server_key: "d6fd09097442b44787fb9a5d76c9096d"
server_key_iv: "af884792ead4d1ed38f96cd65cdd9297"
server_key_version: "encryptionkey@0"
I20260811 15:11:10.479566 20729 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 15:11:10.482129 20749 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:10.483498 20729 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.003s
I20260811 15:11:10.483631 20729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "50c523135e324a059998f8c92d341b97"
format_stamp: "Formatted at 2026-08-11 15:11:10 on dist-test-slave-1vmg"
server_key: "d6fd09097442b44787fb9a5d76c9096d"
server_key_iv: "af884792ead4d1ed38f96cd65cdd9297"
server_key_version: "encryptionkey@0"
I20260811 15:11:10.483750 20729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:10.504204 20729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:10.507889 20729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:10.508093 20729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:10.515486 20729 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:45969
I20260811 15:11:10.515782 20840 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:45969 every 8 connection(s)
I20260811 15:11:10.516639 20729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:11:10.519595 20841 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:10.523504 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 20729
I20260811 15:11:10.523630 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:11:10.523891 30752 external_mini_cluster.cc:1428] Setting key fcd723235e689e6dadd1b0775ce32347
I20260811 15:11:10.526021 20841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97: Bootstrap starting.
I20260811 15:11:10.528918 20841 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97: Neither blocks nor log segments found. Creating new log.
I20260811 15:11:10.529882 20841 log.cc:824] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:10.532241 20841 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97: No bootstrap required, opened a new log
Aug 11 15:11:10 dist-test-slave-1vmg krb5kdc[20707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461070, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:10.537289 20841 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c523135e324a059998f8c92d341b97" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45969 } }
I20260811 15:11:10.537616 20841 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:10.537665 20841 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50c523135e324a059998f8c92d341b97, State: Initialized, Role: FOLLOWER
I20260811 15:11:10.538211 20841 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c523135e324a059998f8c92d341b97" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45969 } }
I20260811 15:11:10.538378 20841 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:11:10.538456 20841 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:11:10.538570 20841 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:10.539855 20841 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c523135e324a059998f8c92d341b97" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45969 } }
I20260811 15:11:10.540257 20841 leader_election.cc:302] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50c523135e324a059998f8c92d341b97; no voters:
I20260811 15:11:10.540668 20841 leader_election.cc:288] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:11:10.540786 20853 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:11:10.541108 20850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:10.525793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56178 (local address 127.30.8.62:45969)
0811 15:11:10.526281 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:10.526303 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:10.526349 (+ 46us) server_negotiation.cc:407] Connection header received
0811 15:11:10.527240 (+ 891us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:10.527266 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:10.527743 (+ 477us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:10.528117 (+ 374us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:10.528826 (+ 709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:10.530347 (+ 1521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:10.531330 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:10.532025 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:10.532189 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:10.534894 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:10.534904 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:10.534957 (+ 53us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:10.534998 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:10.537390 (+ 2392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:10.537898 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:10.537905 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:10.537923 (+ 18us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:10.538090 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:10.538345 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:10.538348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:10.538350 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:10.538958 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:10.539172 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:10.539632 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 15:11:10.539996 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":140,"threads_started":1}
I20260811 15:11:10.541221 20853 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 1 LEADER]: Becoming Leader. State: Replica: 50c523135e324a059998f8c92d341b97, State: Running, Role: LEADER
I20260811 15:11:10.541772 20853 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c523135e324a059998f8c92d341b97" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45969 } }
I20260811 15:11:10.542186 20841 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:11:10.544471 20856 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50c523135e324a059998f8c92d341b97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c523135e324a059998f8c92d341b97" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45969 } } }
I20260811 15:11:10.544629 20856 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [sys.catalog]: This master's current role is: LEADER
I20260811 15:11:10.544749 20857 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50c523135e324a059998f8c92d341b97. Latest consensus state: current_term: 1 leader_uuid: "50c523135e324a059998f8c92d341b97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c523135e324a059998f8c92d341b97" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45969 } } }
I20260811 15:11:10.544855 20857 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [sys.catalog]: This master's current role is: LEADER
I20260811 15:11:10.545109 20875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:11:10.548696 20875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:11:10.554311 20875 catalog_manager.cc:1347] Generated new cluster ID: 0effabe709ff474ca096d5222d750fed
I20260811 15:11:10.554473 20875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:11:10.568640 20875 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:11:10.570039 20875 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:11:10.581943 20875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97: Generated new TSK 0
I20260811 15:11:10.582978 20875 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
I20260811 15:11:10.616520 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45969
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:11:10.735518 20885 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:10.735841 20885 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:10.735908 20885 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:10.740095 20885 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:10.740181 20885 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:10.740264 20885 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:11:10.746006 20885 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20885
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:10.747274 20885 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:10.748346 20885 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:10.758414 20896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:10.758565 20885 server_base.cc:1034] running on GCE node
W20260811 15:11:10.758404 20901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:10.758420 20897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:10.759119 20885 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:10.759949 20885 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:10.761129 20885 hybrid_clock.cc:648] HybridClock initialized: now 1786461070761103 us; error 42 us; skew 500 ppm
Aug 11 15:11:10 dist-test-slave-1vmg krb5kdc[20707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461070, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:10.763973 20885 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:11:10.764958 20885 webserver.cc:460] Webserver started at http://127.30.8.1:40379/ using document root <none> and password file <none>
I20260811 15:11:10.765535 20885 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:10.765615 20885 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:10.765825 20885 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:11:10.767977 20885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "eda565b102f448078de510d0a0e548f0"
format_stamp: "Formatted at 2026-08-11 15:11:10 on dist-test-slave-1vmg"
server_key: "4fca43a9641337d447dfa02283fb492c"
server_key_iv: "5ef529ee9df4a0fb1e2760f42861647d"
server_key_version: "encryptionkey@0"
I20260811 15:11:10.768550 20885 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "eda565b102f448078de510d0a0e548f0"
format_stamp: "Formatted at 2026-08-11 15:11:10 on dist-test-slave-1vmg"
server_key: "4fca43a9641337d447dfa02283fb492c"
server_key_iv: "5ef529ee9df4a0fb1e2760f42861647d"
server_key_version: "encryptionkey@0"
I20260811 15:11:10.772490 20885 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:11:10.775048 20914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:10.776294 20885 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:11:10.776417 20885 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "eda565b102f448078de510d0a0e548f0"
format_stamp: "Formatted at 2026-08-11 15:11:10 on dist-test-slave-1vmg"
server_key: "4fca43a9641337d447dfa02283fb492c"
server_key_iv: "5ef529ee9df4a0fb1e2760f42861647d"
server_key_version: "encryptionkey@0"
I20260811 15:11:10.776525 20885 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:10.796252 20885 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:10.799676 20885 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:10.799894 20885 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:10.800518 20885 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:11:10.801414 20885 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:11:10.801465 20885 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:10.801522 20885 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:11:10.801558 20885 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:10.810369 20885 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:40023
I20260811 15:11:10.810397 21112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:40023 every 8 connection(s)
I20260811 15:11:10.811375 20885 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:11:10.815513 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 20885
I20260811 15:11:10.815656 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:11:10.815917 30752 external_mini_cluster.cc:1428] Setting key 65e069834e391dfe6df58a08a9d16306
Aug 11 15:11:10 dist-test-slave-1vmg krb5kdc[20707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461070, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:10.826295 20850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:10.813289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:51371 (local address 127.30.8.62:45969)
0811 15:11:10.813431 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:10.813435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:10.814280 (+ 845us) server_negotiation.cc:407] Connection header received
0811 15:11:10.815695 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:10.815698 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:10.815758 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:10.815838 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:10.817330 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:10.818082 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:10.819325 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:10.819637 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:10.819721 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:10.822468 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:10.822473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:10.822476 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:10.822506 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:10.824118 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:10.824615 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:10.824618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:10.824620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:10.824716 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:10.825013 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:10.825016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:10.825018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:10.825273 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:10.825381 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:10.825959 (+ 578us) server_negotiation.cc:299] Negotiation successful
0811 15:11:10.826107 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
WARNING: no policy specified for kudu/127.30.8.2@KRBTEST.COM; defaulting to no policy
I20260811 15:11:10.827211 21115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:10.813646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:51371)
0811 15:11:10.814127 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:10.814170 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:10.815469 (+ 1299us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:10.816039 (+ 570us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:10.816049 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:10.816507 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:10.817169 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:10.817190 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:10.818206 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:10.818209 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:10.819180 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:10.819191 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:10.819814 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:10.819818 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:10.820028 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:10.820710 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:10.820732 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:10.822272 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:10.824238 (+ 1966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:10.824243 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:10.824259 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:10.824518 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:10.824798 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:10.824800 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:10.824801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:10.824931 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:10.825365 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:10.825370 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:10.825638 (+ 268us) client_negotiation.cc:769] Sending connection context
0811 15:11:10.825879 (+ 241us) client_negotiation.cc:240] Negotiation successful
0811 15:11:10.826198 (+ 319us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":125,"threads_started":1}
I20260811 15:11:10.828454 21113 heartbeater.cc:347] Connected to a master server at 127.30.8.62:45969
I20260811 15:11:10.828706 21113 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:10.829263 21113 heartbeater.cc:511] Master 127.30.8.62:45969 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.2@KRBTEST.COM" created.
I20260811 15:11:10.830889 20776 ts_manager.cc:194] Registered new tserver with Master: eda565b102f448078de510d0a0e548f0 (127.30.8.1:40023)
I20260811 15:11:10.832369 20776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:51371
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
I20260811 15:11:10.845058 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45969
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:11:10.962276 21120 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:10.962569 21120 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:10.962615 21120 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:10.966651 21120 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:10.966732 21120 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:10.966791 21120 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:11:10.972332 21120 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21120
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:10.973469 21120 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:10.974462 21120 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:10.984318 21127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:10.984325 21130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:10.984336 21128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:10.984678 21120 server_base.cc:1034] running on GCE node
I20260811 15:11:10.985144 21120 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:10.985849 21120 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:10.986971 21120 hybrid_clock.cc:648] HybridClock initialized: now 1786461070986948 us; error 37 us; skew 500 ppm
Aug 11 15:11:10 dist-test-slave-1vmg krb5kdc[20707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461070, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:10.989806 21120 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:11:10.990751 21120 webserver.cc:460] Webserver started at http://127.30.8.2:38361/ using document root <none> and password file <none>
I20260811 15:11:10.991357 21120 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:10.991431 21120 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:10.991643 21120 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:11:10.993743 21120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c"
format_stamp: "Formatted at 2026-08-11 15:11:10 on dist-test-slave-1vmg"
server_key: "1c5a5c4d46bdf0e8926c0d702855e20b"
server_key_iv: "b41602f62a71ff5ec820b83cbfc84aa2"
server_key_version: "encryptionkey@0"
I20260811 15:11:10.994289 21120 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c"
format_stamp: "Formatted at 2026-08-11 15:11:10 on dist-test-slave-1vmg"
server_key: "1c5a5c4d46bdf0e8926c0d702855e20b"
server_key_iv: "b41602f62a71ff5ec820b83cbfc84aa2"
server_key_version: "encryptionkey@0"
I20260811 15:11:10.998129 21120 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 15:11:11.000692 21143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:11.001959 21120 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 15:11:11.002094 21120 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c"
format_stamp: "Formatted at 2026-08-11 15:11:10 on dist-test-slave-1vmg"
server_key: "1c5a5c4d46bdf0e8926c0d702855e20b"
server_key_iv: "b41602f62a71ff5ec820b83cbfc84aa2"
server_key_version: "encryptionkey@0"
I20260811 15:11:11.002204 21120 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:11.011534 21120 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:11.014410 21120 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:11.014614 21120 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:11.015281 21120 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:11:11.016163 21120 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:11:11.016213 21120 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:11.016268 21120 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:11:11.016297 21120 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:11.025111 21120 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:41153
I20260811 15:11:11.025131 21334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:41153 every 8 connection(s)
I20260811 15:11:11.026079 21120 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:11:11.032333 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 21120
I20260811 15:11:11.032481 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:11:11.032773 30752 external_mini_cluster.cc:1428] Setting key 367076676c97dac2b846275a027fc821
Aug 11 15:11:11 dist-test-slave-1vmg krb5kdc[20707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461070, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:11.040373 20850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.028012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:39105 (local address 127.30.8.62:45969)
0811 15:11:11.028163 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:11.028167 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:11.029000 (+ 833us) server_negotiation.cc:407] Connection header received
0811 15:11:11.030309 (+ 1309us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:11.030312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:11.030361 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:11.030435 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:11.031705 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.032332 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.033608 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.033907 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.034002 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:11.036703 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:11.036708 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:11.036711 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:11.036742 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:11.038161 (+ 1419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:11.038672 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:11.038675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:11.038676 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:11.038757 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:11.039135 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:11.039139 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:11.039140 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:11.039341 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:11.039432 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:11.040049 (+ 617us) server_negotiation.cc:299] Negotiation successful
0811 15:11:11.040191 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 15:11:11.041266 21337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.028383 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:39105)
0811 15:11:11.028840 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:11.028884 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:11.030108 (+ 1224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:11.030600 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:11.030607 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:11.031051 (+ 444us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:11.031578 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:11.031593 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.032450 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.032453 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:11.033492 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:11.033503 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.034061 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.034066 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:11.034236 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:11.034860 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:11.034876 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:11.036512 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:11.038273 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:11.038279 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:11.038294 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:11.038563 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:11.038846 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:11.038849 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:11.038852 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:11.039042 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:11.039446 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:11.039453 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:11.039752 (+ 299us) client_negotiation.cc:769] Sending connection context
0811 15:11:11.039953 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 15:11:11.040330 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":111,"threads_started":1}
I20260811 15:11:11.042495 21335 heartbeater.cc:347] Connected to a master server at 127.30.8.62:45969
I20260811 15:11:11.042752 21335 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:11.043392 21335 heartbeater.cc:511] Master 127.30.8.62:45969 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.8.3@KRBTEST.COM; defaulting to no policy
I20260811 15:11:11.044533 20776 ts_manager.cc:194] Registered new tserver with Master: e393586d25cb4b19a6ffdcc9e9f2a55c (127.30.8.2:41153)
I20260811 15:11:11.045289 20776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:39105
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
I20260811 15:11:11.061981 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45969
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:11:11.178879 21340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:11.179209 21340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:11.179265 21340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:11.183688 21340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:11.183768 21340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:11.183843 21340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:11:11.189498 21340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:11.190662 21340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:11.191767 21340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:11.201578 21352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:11.201685 21340 server_base.cc:1034] running on GCE node
W20260811 15:11:11.201545 21350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:11.201519 21347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:11.202308 21340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:11.203027 21340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:11.204180 21340 hybrid_clock.cc:648] HybridClock initialized: now 1786461071204153 us; error 45 us; skew 500 ppm
Aug 11 15:11:11 dist-test-slave-1vmg krb5kdc[20707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461071, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:11.207024 21340 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:11:11.207993 21340 webserver.cc:460] Webserver started at http://127.30.8.3:37021/ using document root <none> and password file <none>
I20260811 15:11:11.208562 21340 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:11.208629 21340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:11.208886 21340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:11:11.211046 21340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "2ca3edc257ee4a9bb26aa80b674385ba"
format_stamp: "Formatted at 2026-08-11 15:11:11 on dist-test-slave-1vmg"
server_key: "3e658ad93189878ba6e904db79dc7f83"
server_key_iv: "aa9d4215290f107946340210f39ed056"
server_key_version: "encryptionkey@0"
I20260811 15:11:11.211601 21340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "2ca3edc257ee4a9bb26aa80b674385ba"
format_stamp: "Formatted at 2026-08-11 15:11:11 on dist-test-slave-1vmg"
server_key: "3e658ad93189878ba6e904db79dc7f83"
server_key_iv: "aa9d4215290f107946340210f39ed056"
server_key_version: "encryptionkey@0"
I20260811 15:11:11.215395 21340 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:11:11.217866 21360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:11.219164 21340 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:11:11.219285 21340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "2ca3edc257ee4a9bb26aa80b674385ba"
format_stamp: "Formatted at 2026-08-11 15:11:11 on dist-test-slave-1vmg"
server_key: "3e658ad93189878ba6e904db79dc7f83"
server_key_iv: "aa9d4215290f107946340210f39ed056"
server_key_version: "encryptionkey@0"
I20260811 15:11:11.219393 21340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:11.248279 21340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:11.251276 21340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:11.251473 21340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:11.252092 21340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:11:11.252981 21340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:11:11.253031 21340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:11.253084 21340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:11:11.253113 21340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:11.262180 21340 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:43089
I20260811 15:11:11.262204 21570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:43089 every 8 connection(s)
I20260811 15:11:11.263201 21340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:11:11.269995 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 21340
I20260811 15:11:11.270139 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:11:11.270442 30752 external_mini_cluster.cc:1428] Setting key 144fa0f31ba3ada18cc32ef153f655a9
Aug 11 15:11:11 dist-test-slave-1vmg krb5kdc[20707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461071, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:11.276752 20850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.265054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:56823 (local address 127.30.8.62:45969)
0811 15:11:11.265240 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:11.265245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:11.266093 (+ 848us) server_negotiation.cc:407] Connection header received
0811 15:11:11.267467 (+ 1374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:11.267470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:11.267520 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:11.267592 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:11.268750 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.269277 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.270722 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.271047 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.271143 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:11.273677 (+ 2534us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:11.273680 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:11.273682 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:11.273710 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:11.274996 (+ 1286us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:11.275468 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:11.275470 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:11.275471 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:11.275545 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:11.275809 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:11.275811 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:11.275812 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:11.275983 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:11.276054 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:11.276514 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 15:11:11.276616 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 15:11:11.277673 21574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.265457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:56823)
0811 15:11:11.265936 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:11.265980 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:11.267268 (+ 1288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:11.267743 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:11.267751 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:11.268131 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:11.268633 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:11.268646 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.269368 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.269371 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:11.270605 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:11.270619 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.271201 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.271206 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:11.271401 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:11.272042 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:11.272060 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:11.273517 (+ 1457us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:11.275100 (+ 1583us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:11.275106 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:11.275121 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:11.275383 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:11.275606 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:11.275608 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:11.275609 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:11.275744 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:11.276057 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:11.276062 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:11.276318 (+ 256us) client_negotiation.cc:769] Sending connection context
0811 15:11:11.276485 (+ 167us) client_negotiation.cc:240] Negotiation successful
0811 15:11:11.276771 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":140,"threads_started":1}
I20260811 15:11:11.278782 21571 heartbeater.cc:347] Connected to a master server at 127.30.8.62:45969
I20260811 15:11:11.279065 21571 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:11.279597 21571 heartbeater.cc:511] Master 127.30.8.62:45969 requested a full tablet report, sending...
I20260811 15:11:11.280712 20776 ts_manager.cc:194] Registered new tserver with Master: 2ca3edc257ee4a9bb26aa80b674385ba (127.30.8.3:43089)
I20260811 15:11:11.281271 20776 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:56823
I20260811 15:11:11.284353 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 15:11:11.293392 20850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.286310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56182 (local address 127.30.8.62:45969)
0811 15:11:11.286436 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:11.286441 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:11.286599 (+ 158us) server_negotiation.cc:407] Connection header received
0811 15:11:11.286732 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:11.286734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:11.286774 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:11.286834 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:11.287626 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.288138 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.288979 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.289191 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.289277 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:11.290598 (+ 1321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:11.290601 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:11.290603 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:11.290626 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:11.291796 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:11.292179 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:11.292182 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:11.292184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:11.292260 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:11.292498 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:11.292502 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:11.292504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:11.292744 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:11.292892 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:11.293110 (+ 218us) server_negotiation.cc:299] Negotiation successful
0811 15:11:11.293225 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 15:11:11.296334 20776 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56182:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:11:11.299471 20776 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:11:11.314121 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.308178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:41153 (local address 127.0.0.1:47552)
0811 15:11:11.308746 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:11.308760 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:11.308899 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:11.310041 (+ 1142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:11.310044 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:11.310063 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:11.310149 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:11.310155 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.311215 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.311218 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:11.312751 (+ 1533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:11.312760 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.313541 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.313545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:11.313669 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:11.313710 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 15:11:11.313823 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 15:11:11.313960 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"mutex_wait_us":59,"spinlock_wait_cycles":9600,"thread_start_us":109,"threads_started":1}
I20260811 15:11:11.314126 21599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.308508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47552 (local address 127.30.8.2:41153)
0811 15:11:11.309631 (+ 1123us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:11.309643 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:11.309665 (+ 22us) server_negotiation.cc:407] Connection header received
0811 15:11:11.309733 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:11.309737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:11.309915 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:11.310065 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:11.310281 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.311094 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.312844 (+ 1750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.313414 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.313513 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:11.313691 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:11.313870 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 15:11:11.313969 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1023,"thread_start_us":131,"threads_started":1}
I20260811 15:11:11.314482 21605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.309188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57164 (local address 127.30.8.3:43089)
0811 15:11:11.309673 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:11.309686 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:11.309709 (+ 23us) server_negotiation.cc:407] Connection header received
0811 15:11:11.309793 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:11.309798 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:11.309998 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:11.310147 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:11.310449 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.311476 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.312781 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.313598 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.313702 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:11.313834 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:11.314262 (+ 428us) server_negotiation.cc:299] Negotiation successful
0811 15:11:11.314355 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":150,"threads_started":1}
I20260811 15:11:11.314414 21592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.308071 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:40023 (local address 127.0.0.1:52586)
0811 15:11:11.308705 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:11.308757 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:11.308899 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:11.309682 (+ 783us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:11.309689 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:11.309713 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:11.309851 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:11.309858 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.311296 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.311299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:11.312662 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:11.312674 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.313900 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.313904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:11.314062 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:11.314112 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 15:11:11.314183 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 15:11:11.314274 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":89,"spinlock_wait_cycles":3840,"thread_start_us":70,"threads_started":1}
I20260811 15:11:11.314767 21596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.308196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52586 (local address 127.30.8.1:40023)
0811 15:11:11.308948 (+ 752us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:11.308970 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:11.308994 (+ 24us) server_negotiation.cc:407] Connection header received
0811 15:11:11.309073 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:11.309079 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:11.309297 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:11.309453 (+ 156us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:11.309963 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.311168 (+ 1205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.312778 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.313598 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.313702 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:11.313834 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:11.314528 (+ 694us) server_negotiation.cc:299] Negotiation successful
0811 15:11:11.314635 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":648,"thread_start_us":114,"threads_started":1}
I20260811 15:11:11.315063 21602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.308769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:43089 (local address 127.0.0.1:57164)
0811 15:11:11.309304 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:11.309318 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:11.309401 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:11.310237 (+ 836us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:11.310240 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:11.310256 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:11.310339 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:11.310345 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.311601 (+ 1256us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.311604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:11.312647 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:11.312658 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.313900 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.313904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:11.314114 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:11.314132 (+ 18us) client_negotiation.cc:769] Sending connection context
0811 15:11:11.314188 (+ 56us) client_negotiation.cc:240] Negotiation successful
0811 15:11:11.314909 (+ 721us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":474,"mutex_wait_us":704,"thread_start_us":104,"threads_started":1}
I20260811 15:11:11.317952 20990 tablet_service.cc:1467] Processing CreateTablet for tablet ca1527df3d034a279eaaf3fcd4428d9a (DEFAULT_TABLE table=test-table [id=ae5de5e561bf4cfaad930b6ebb5c7638]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:11:11.317952 21220 tablet_service.cc:1467] Processing CreateTablet for tablet ca1527df3d034a279eaaf3fcd4428d9a (DEFAULT_TABLE table=test-table [id=ae5de5e561bf4cfaad930b6ebb5c7638]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:11:11.319253 21220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca1527df3d034a279eaaf3fcd4428d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:11.319253 20990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca1527df3d034a279eaaf3fcd4428d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:11.321059 21457 tablet_service.cc:1467] Processing CreateTablet for tablet ca1527df3d034a279eaaf3fcd4428d9a (DEFAULT_TABLE table=test-table [id=ae5de5e561bf4cfaad930b6ebb5c7638]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:11:11.321990 21457 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca1527df3d034a279eaaf3fcd4428d9a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:11.328626 21609 tablet_bootstrap.cc:492] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0: Bootstrap starting.
I20260811 15:11:11.329413 21610 tablet_bootstrap.cc:492] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba: Bootstrap starting.
I20260811 15:11:11.329471 21608 tablet_bootstrap.cc:492] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c: Bootstrap starting.
I20260811 15:11:11.331486 21609 tablet_bootstrap.cc:654] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0: Neither blocks nor log segments found. Creating new log.
I20260811 15:11:11.332156 21608 tablet_bootstrap.cc:654] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c: Neither blocks nor log segments found. Creating new log.
I20260811 15:11:11.332350 21610 tablet_bootstrap.cc:654] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba: Neither blocks nor log segments found. Creating new log.
I20260811 15:11:11.332566 21609 log.cc:824] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:11.333150 21608 log.cc:824] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:11.334336 21610 log.cc:824] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:11.335493 21609 tablet_bootstrap.cc:492] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0: No bootstrap required, opened a new log
I20260811 15:11:11.335736 21609 ts_tablet_manager.cc:1397] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 15:11:11.335980 21608 tablet_bootstrap.cc:492] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c: No bootstrap required, opened a new log
I20260811 15:11:11.336186 21608 ts_tablet_manager.cc:1397] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 15:11:11.336246 21610 tablet_bootstrap.cc:492] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba: No bootstrap required, opened a new log
I20260811 15:11:11.336414 21610 ts_tablet_manager.cc:1397] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 15:11:11.339187 21609 raft_consensus.cc:348] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } } peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } }
I20260811 15:11:11.339426 21609 raft_consensus.cc:374] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:11.339488 21609 raft_consensus.cc:729] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda565b102f448078de510d0a0e548f0, State: Initialized, Role: FOLLOWER
I20260811 15:11:11.339958 21609 consensus_queue.cc:260] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } } peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } }
I20260811 15:11:11.340024 21608 raft_consensus.cc:348] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } } peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } }
I20260811 15:11:11.340024 21610 raft_consensus.cc:348] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } } peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } }
I20260811 15:11:11.340276 21610 raft_consensus.cc:374] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:11.340276 21608 raft_consensus.cc:374] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:11.340334 21610 raft_consensus.cc:729] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ca3edc257ee4a9bb26aa80b674385ba, State: Initialized, Role: FOLLOWER
I20260811 15:11:11.340341 21608 raft_consensus.cc:729] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e393586d25cb4b19a6ffdcc9e9f2a55c, State: Initialized, Role: FOLLOWER
I20260811 15:11:11.340749 21608 consensus_queue.cc:260] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } } peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } }
I20260811 15:11:11.340749 21610 consensus_queue.cc:260] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } } peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } }
I20260811 15:11:11.340989 21609 ts_tablet_manager.cc:1428] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 15:11:11.341388 21113 heartbeater.cc:503] Master 127.30.8.62:45969 was elected leader, sending a full tablet report...
I20260811 15:11:11.341665 21608 ts_tablet_manager.cc:1428] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 15:11:11.341647 21571 heartbeater.cc:503] Master 127.30.8.62:45969 was elected leader, sending a full tablet report...
I20260811 15:11:11.341665 21610 ts_tablet_manager.cc:1428] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba: Time spent starting tablet: real 0.005s user 0.000s sys 0.005s
I20260811 15:11:11.342263 21335 heartbeater.cc:503] Master 127.30.8.62:45969 was elected leader, sending a full tablet report...
I20260811 15:11:11.376358 21620 raft_consensus.cc:482] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:11:11.376592 21620 raft_consensus.cc:504] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } } peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } }
I20260811 15:11:11.377801 21620 leader_election.cc:288] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eda565b102f448078de510d0a0e548f0 (127.30.8.1:40023), 2ca3edc257ee4a9bb26aa80b674385ba (127.30.8.3:43089)
I20260811 15:11:11.381666 21337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.378032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:40023 (local address 127.30.8.2:40257)
0811 15:11:11.378163 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:11.378179 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:11.378273 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:11.378564 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:11.378567 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:11.378586 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:11.378717 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:11.378723 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.379504 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.379508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:11.380415 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:11.380426 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.381210 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.381214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:11.381323 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:11.381351 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 15:11:11.381435 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 15:11:11.381512 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 15:11:11.381737 21596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.378177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:40257 (local address 127.30.8.1:40023)
0811 15:11:11.378300 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:11.378305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:11.378321 (+ 16us) server_negotiation.cc:407] Connection header received
0811 15:11:11.378384 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:11.378387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:11.378440 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:11.378531 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:11.378833 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.379398 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.380533 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.381084 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.381201 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:11.381263 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:11.381530 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 15:11:11.381605 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 15:11:11.381889 21624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.378335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:43089 (local address 127.30.8.2:47949)
0811 15:11:11.378721 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:11.378731 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:11.378820 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:11.379049 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:11.379052 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:11.379066 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:11.379174 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:11.379179 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.379885 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.379888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:11.380773 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:11.380784 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.381496 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.381500 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:11.381599 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:11.381633 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 15:11:11.381705 (+ 72us) client_negotiation.cc:240] Negotiation successful
0811 15:11:11.381767 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":120,"threads_started":1}
I20260811 15:11:11.381920 21605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.378446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:47949 (local address 127.30.8.3:43089)
0811 15:11:11.378593 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:11.378598 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:11.378803 (+ 205us) server_negotiation.cc:407] Connection header received
0811 15:11:11.378904 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:11.378906 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:11.378968 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:11.379029 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:11.379269 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.379783 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.380894 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.381364 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.381470 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:11.381528 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:11.381740 (+ 212us) server_negotiation.cc:299] Negotiation successful
0811 15:11:11.381812 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 15:11:11.382490 21032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca1527df3d034a279eaaf3fcd4428d9a" candidate_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda565b102f448078de510d0a0e548f0" is_pre_election: true
I20260811 15:11:11.382622 21492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca1527df3d034a279eaaf3fcd4428d9a" candidate_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" is_pre_election: true
I20260811 15:11:11.382817 21032 raft_consensus.cc:2454] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e393586d25cb4b19a6ffdcc9e9f2a55c in term 0.
I20260811 15:11:11.382890 21492 raft_consensus.cc:2454] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e393586d25cb4b19a6ffdcc9e9f2a55c in term 0.
I20260811 15:11:11.383343 21145 leader_election.cc:302] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e393586d25cb4b19a6ffdcc9e9f2a55c, eda565b102f448078de510d0a0e548f0; no voters:
I20260811 15:11:11.383625 21620 raft_consensus.cc:2781] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:11:11.383705 21620 raft_consensus.cc:482] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:11:11.383736 21620 raft_consensus.cc:3037] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:11.384862 21620 raft_consensus.cc:504] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } } peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } }
I20260811 15:11:11.385205 21620 leader_election.cc:288] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [CANDIDATE]: Term 1 election: Requested vote from peers eda565b102f448078de510d0a0e548f0 (127.30.8.1:40023), 2ca3edc257ee4a9bb26aa80b674385ba (127.30.8.3:43089)
I20260811 15:11:11.385600 21032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca1527df3d034a279eaaf3fcd4428d9a" candidate_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eda565b102f448078de510d0a0e548f0"
I20260811 15:11:11.385627 21492 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca1527df3d034a279eaaf3fcd4428d9a" candidate_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ca3edc257ee4a9bb26aa80b674385ba"
I20260811 15:11:11.385730 21492 raft_consensus.cc:3037] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:11.385725 21032 raft_consensus.cc:3037] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:11.386812 21492 raft_consensus.cc:2454] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e393586d25cb4b19a6ffdcc9e9f2a55c in term 1.
I20260811 15:11:11.386817 21032 raft_consensus.cc:2454] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e393586d25cb4b19a6ffdcc9e9f2a55c in term 1.
I20260811 15:11:11.387112 21145 leader_election.cc:302] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e393586d25cb4b19a6ffdcc9e9f2a55c, eda565b102f448078de510d0a0e548f0; no voters:
I20260811 15:11:11.387288 21620 raft_consensus.cc:2781] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:11:11.387514 21620 raft_consensus.cc:686] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [term 1 LEADER]: Becoming Leader. State: Replica: e393586d25cb4b19a6ffdcc9e9f2a55c, State: Running, Role: LEADER
I20260811 15:11:11.387918 21620 consensus_queue.cc:237] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } } peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } }
I20260811 15:11:11.391247 20773 catalog_manager.cc:5495] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c reported cstate change: term changed from 0 to 1, leader changed from <none> to e393586d25cb4b19a6ffdcc9e9f2a55c (127.30.8.2). New cstate: current_term: 1 leader_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } health_report { overall_health: UNKNOWN } } }
I20260811 15:11:11.418365 21599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.415178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47560 (local address 127.30.8.2:41153)
0811 15:11:11.415300 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:11.415303 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:11.415316 (+ 13us) server_negotiation.cc:407] Connection header received
0811 15:11:11.415396 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:11.415399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:11.415457 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:11.415558 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:11:11.415733 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.416282 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.417078 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.417280 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.417351 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:11.417579 (+ 228us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:11:11.418032 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:11:11.418123 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:11.418171 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 15:11:11.418225 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 15:11:11.424559 21492 raft_consensus.cc:1261] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba [term 1 FOLLOWER]: Refusing update from remote peer e393586d25cb4b19a6ffdcc9e9f2a55c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:11:11.424551 21032 raft_consensus.cc:1261] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0 [term 1 FOLLOWER]: Refusing update from remote peer e393586d25cb4b19a6ffdcc9e9f2a55c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:11:11.425336 21620 consensus_queue.cc:1035] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [LEADER]: Connected to new peer: Peer: permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:11:11.425540 21626 consensus_queue.cc:1035] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:11:11.433837 21631 mvcc.cc:205] Tried to move back new op lower bound from 7317344548547137536 to 7317344548407939072. Current Snapshot: MvccSnapshot[applied={T|T < 7317344548547137536}]
I20260811 15:11:11.443140 20776 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:11:11.443379 20776 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56182:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:11:11.446261 20776 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97: Sending DeleteTablet for 3 replicas of tablet ca1527df3d034a279eaaf3fcd4428d9a
I20260811 15:11:11.447178 21220 tablet_service.cc:1514] Processing DeleteTablet for tablet ca1527df3d034a279eaaf3fcd4428d9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:11:11 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:47552
I20260811 15:11:11.447240 20990 tablet_service.cc:1514] Processing DeleteTablet for tablet ca1527df3d034a279eaaf3fcd4428d9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:11:11 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:52586
I20260811 15:11:11.447465 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 20885
I20260811 15:11:11.447747 21457 tablet_service.cc:1514] Processing DeleteTablet for tablet ca1527df3d034a279eaaf3fcd4428d9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:11:11 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:57164
I20260811 15:11:11.458858 21650 tablet_replica.cc:330] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c: stopping tablet replica
I20260811 15:11:11.458853 21651 tablet_replica.cc:330] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba: stopping tablet replica
I20260811 15:11:11.459353 21651 raft_consensus.cc:2229] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:11:11.459353 21650 raft_consensus.cc:2229] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:11:11.459806 21651 raft_consensus.cc:2258] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:11:11.459959 21650 raft_consensus.cc:2258] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:11:11.464766 21650 ts_tablet_manager.cc:1905] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:11:11.464766 21651 ts_tablet_manager.cc:1905] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:11:11.467775 21650 ts_tablet_manager.cc:1918] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:11:11.467816 21651 ts_tablet_manager.cc:1918] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:11:11.467919 21651 log.cc:1196] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/ca1527df3d034a279eaaf3fcd4428d9a
I20260811 15:11:11.467919 21650 log.cc:1196] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/ca1527df3d034a279eaaf3fcd4428d9a
I20260811 15:11:11.468251 21651 ts_tablet_manager.cc:1939] T ca1527df3d034a279eaaf3fcd4428d9a P 2ca3edc257ee4a9bb26aa80b674385ba: Deleting consensus metadata
I20260811 15:11:11.468281 21650 ts_tablet_manager.cc:1939] T ca1527df3d034a279eaaf3fcd4428d9a P e393586d25cb4b19a6ffdcc9e9f2a55c: Deleting consensus metadata
I20260811 15:11:11.469681 20757 catalog_manager.cc:4841] TS e393586d25cb4b19a6ffdcc9e9f2a55c (127.30.8.2:41153): tablet ca1527df3d034a279eaaf3fcd4428d9a (table test-table [id=ae5de5e561bf4cfaad930b6ebb5c7638]) successfully deleted
I20260811 15:11:11.469681 20755 catalog_manager.cc:4841] TS 2ca3edc257ee4a9bb26aa80b674385ba (127.30.8.3:43089): tablet ca1527df3d034a279eaaf3fcd4428d9a (table test-table [id=ae5de5e561bf4cfaad930b6ebb5c7638]) successfully deleted
I20260811 15:11:11.471429 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 21120
W20260811 15:11:11.471454 20755 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.8.1:40023 (error 108)
I20260811 15:11:11.477669 21593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.477310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:40023 (local address 127.0.0.1:52594)
0811 15:11:11.477472 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:11.477560 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.1:40023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 15:11:11.477886 20755 catalog_manager.cc:4584] TS eda565b102f448078de510d0a0e548f0 (127.30.8.1:40023): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ca1527df3d034a279eaaf3fcd4428d9a: Network error: Client connection negotiation failed: client connection to 127.30.8.1:40023: connect: Connection refused (error 111)
I20260811 15:11:11.482975 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 21340
I20260811 15:11:11.494138 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 20729
I20260811 15:11:11.509809 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:45969
--webserver_interface=127.30.8.62
--webserver_port=43883
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:45969
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:11:11.627269 21654 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:11.627578 21654 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:11.627636 21654 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:11.631826 21654 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:11:11.631914 21654 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:11.631942 21654 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:11.631964 21654 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:11:11.631992 21654 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:11:11.637653 21654 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:45969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:45969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=43883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21654
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:11.638854 21654 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:11.639995 21654 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:11.654661 21670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:11.654667 21674 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:11.654661 21671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:11.654729 21654 server_base.cc:1034] running on GCE node
I20260811 15:11:11.655687 21654 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:11.657153 21654 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:11.658344 21654 hybrid_clock.cc:648] HybridClock initialized: now 1786461071658323 us; error 36 us; skew 500 ppm
Aug 11 15:11:11 dist-test-slave-1vmg krb5kdc[20707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461071, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:11.661341 21654 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:11:11.662344 21654 webserver.cc:460] Webserver started at http://127.30.8.62:43883/ using document root <none> and password file <none>
I20260811 15:11:11.663002 21654 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:11.663087 21654 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:11.667292 21654 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 15:11:11.669796 21685 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:11.671126 21654 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:11:11.671254 21654 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "50c523135e324a059998f8c92d341b97"
format_stamp: "Formatted at 2026-08-11 15:11:10 on dist-test-slave-1vmg"
server_key: "d6fd09097442b44787fb9a5d76c9096d"
server_key_iv: "af884792ead4d1ed38f96cd65cdd9297"
server_key_version: "encryptionkey@0"
I20260811 15:11:11.671898 21654 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:11.696097 21654 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:11.699208 21654 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:11.699410 21654 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:11.706789 21654 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:45969
I20260811 15:11:11.707101 21777 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:45969 every 8 connection(s)
I20260811 15:11:11.708043 21654 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:11:11.712698 21778 sys_catalog.cc:263] Verifying existing consensus state
I20260811 15:11:11.714564 21778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97: Bootstrap starting.
I20260811 15:11:11.717401 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 21654
I20260811 15:11:11.717855 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:40023
--local_ip_for_outbound_sockets=127.30.8.1
--tserver_master_addrs=127.30.8.62:45969
--webserver_port=40379
--webserver_interface=127.30.8.1
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:11:11.732156 21778 log.cc:824] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:11.741629 21778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 15:11:11.742027 21778 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97: Bootstrap complete.
I20260811 15:11:11.746150 21778 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c523135e324a059998f8c92d341b97" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45969 } }
I20260811 15:11:11.746568 21778 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50c523135e324a059998f8c92d341b97, State: Initialized, Role: FOLLOWER
I20260811 15:11:11.747097 21778 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c523135e324a059998f8c92d341b97" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45969 } }
I20260811 15:11:11.747241 21778 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:11:11.747320 21778 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:11:11.747423 21778 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 1 FOLLOWER]: Advancing to term 2
I20260811 15:11:11.749437 21778 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c523135e324a059998f8c92d341b97" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45969 } }
I20260811 15:11:11.749782 21778 leader_election.cc:302] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50c523135e324a059998f8c92d341b97; no voters:
I20260811 15:11:11.750131 21778 leader_election.cc:288] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 15:11:11.750268 21789 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 15:11:11.750475 21789 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 2 LEADER]: Becoming Leader. State: Replica: 50c523135e324a059998f8c92d341b97, State: Running, Role: LEADER
I20260811 15:11:11.750797 21789 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c523135e324a059998f8c92d341b97" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45969 } }
I20260811 15:11:11.751482 21778 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:11:11.752341 21790 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "50c523135e324a059998f8c92d341b97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c523135e324a059998f8c92d341b97" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45969 } } }
I20260811 15:11:11.752468 21790 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [sys.catalog]: This master's current role is: LEADER
I20260811 15:11:11.752782 21792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50c523135e324a059998f8c92d341b97. Latest consensus state: current_term: 2 leader_uuid: "50c523135e324a059998f8c92d341b97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c523135e324a059998f8c92d341b97" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45969 } } }
I20260811 15:11:11.752868 21792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [sys.catalog]: This master's current role is: LEADER
I20260811 15:11:11.756839 21814 catalog_manager.cc:1259] Loaded cluster ID: 0effabe709ff474ca096d5222d750fed
I20260811 15:11:11.756910 21814 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97: loading cluster ID for follower catalog manager: success
W20260811 15:11:11.757926 21814 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260811 15:11:11.758006 21814 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260811 15:11:11.758073 21802 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:11:11.760627 21802 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:11:11.761138 21802 catalog_manager.cc:1259] Loaded cluster ID: 0effabe709ff474ca096d5222d750fed
I20260811 15:11:11.761198 21802 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:11:11.762308 21802 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 15:11:11.767204 21802 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 15:11:11.773234 21802 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:11:11.774127 21802 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97: Loaded TSK: 0
I20260811 15:11:11.775439 21802 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 15:11:11.839820 21781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:11.840129 21781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:11.840205 21781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:11.844358 21781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:11.844453 21781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:11.844548 21781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:11:11.850078 21781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:40023
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=40379
--tserver_master_addrs=127.30.8.62:45969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:11.851462 21781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:11.852530 21781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:11.862712 21826 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:11.862712 21827 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:11.863090 21781 server_base.cc:1034] running on GCE node
W20260811 15:11:11.863200 21830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:11.863734 21781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:11.864605 21781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:11.865823 21781 hybrid_clock.cc:648] HybridClock initialized: now 1786461071865791 us; error 44 us; skew 500 ppm
Aug 11 15:11:11 dist-test-slave-1vmg krb5kdc[20707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461071, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:11.873265 21781 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:11:11.874619 21781 webserver.cc:460] Webserver started at http://127.30.8.1:40379/ using document root <none> and password file <none>
I20260811 15:11:11.875463 21781 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:11.875581 21781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:11.882586 21781 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 15:11:11.885540 21840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:11.887123 21781 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 15:11:11.887288 21781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "eda565b102f448078de510d0a0e548f0"
format_stamp: "Formatted at 2026-08-11 15:11:10 on dist-test-slave-1vmg"
server_key: "4fca43a9641337d447dfa02283fb492c"
server_key_iv: "5ef529ee9df4a0fb1e2760f42861647d"
server_key_version: "encryptionkey@0"
I20260811 15:11:11.888046 21781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:11.913538 21781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:11.916591 21781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:11.916774 21781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:11.917353 21781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:11:11.918497 21851 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 15:11:11.922947 21781 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 15:11:11.923024 21781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.001s
I20260811 15:11:11.923081 21781 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 15:11:11.924767 21781 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 15:11:11.924816 21781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 15:11:11.925089 21851 tablet_bootstrap.cc:492] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0: Bootstrap starting.
I20260811 15:11:11.934976 21781 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:40023
I20260811 15:11:11.935019 22042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:40023 every 8 connection(s)
I20260811 15:11:11.936017 21781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:11:11.936479 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 21781
I20260811 15:11:11.936951 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:41153
--local_ip_for_outbound_sockets=127.30.8.2
--tserver_master_addrs=127.30.8.62:45969
--webserver_port=38361
--webserver_interface=127.30.8.2
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:11:11.943220 21851 log.cc:824] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0: Log is configured to *not* fsync() on all Append() calls
Aug 11 15:11:11 dist-test-slave-1vmg krb5kdc[20707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461071, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:11.952436 21851 tablet_bootstrap.cc:492] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 15:11:11.952893 21851 tablet_bootstrap.cc:492] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0: Bootstrap complete.
I20260811 15:11:11.954252 21851 ts_tablet_manager.cc:1397] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0: Time spent bootstrapping tablet: real 0.029s user 0.026s sys 0.000s
I20260811 15:11:11.954660 22049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.938774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:54381)
0811 15:11:11.940384 (+ 1610us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:11.940443 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:11.941254 (+ 811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:11.942596 (+ 1342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:11.942605 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:11.943166 (+ 561us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:11.943844 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:11.943866 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.945210 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.945213 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:11.946067 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:11.946076 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.946792 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.946796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:11.947001 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:11.947573 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:11.947598 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:11.949041 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:11.951074 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:11.951080 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:11.951094 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:11.951429 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:11.951862 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:11.951864 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:11.951866 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:11.952047 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:11.952725 (+ 678us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:11.952731 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:11.953095 (+ 364us) client_negotiation.cc:769] Sending connection context
0811 15:11:11.953302 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 15:11:11.953666 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1379,"thread_start_us":155,"threads_started":1}
I20260811 15:11:11.954660 22046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.938812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:54381 (local address 127.30.8.62:45969)
0811 15:11:11.940063 (+ 1251us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:11.940088 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:11.940614 (+ 526us) server_negotiation.cc:407] Connection header received
0811 15:11:11.941718 (+ 1104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:11.941733 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:11.942133 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:11.942506 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:11.944057 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.945033 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.946194 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.946651 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.946807 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:11.949248 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:11.949254 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:11.949266 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:11.949295 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:11.950936 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:11.951564 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:11.951571 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:11.951586 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:11.951704 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:11.952160 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:11.952163 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:11.952165 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:11.952537 (+ 372us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:11.952669 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:11.953450 (+ 781us) server_negotiation.cc:299] Negotiation successful
0811 15:11:11.953917 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1085,"thread_start_us":144,"threads_started":1}
I20260811 15:11:11.957288 22043 heartbeater.cc:347] Connected to a master server at 127.30.8.62:45969
I20260811 15:11:11.957566 22043 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:11.957283 21851 raft_consensus.cc:348] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } } peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } }
I20260811 15:11:11.957823 21851 raft_consensus.cc:729] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda565b102f448078de510d0a0e548f0, State: Initialized, Role: FOLLOWER
I20260811 15:11:11.958127 22043 heartbeater.cc:511] Master 127.30.8.62:45969 requested a full tablet report, sending...
I20260811 15:11:11.958329 21851 consensus_queue.cc:260] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } } peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } }
I20260811 15:11:11.959452 21851 ts_tablet_manager.cc:1428] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0: Time spent starting tablet: real 0.005s user 0.004s sys 0.004s
I20260811 15:11:11.961050 21716 ts_manager.cc:194] Registered new tserver with Master: eda565b102f448078de510d0a0e548f0 (127.30.8.1:40023)
I20260811 15:11:11.965181 21716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:54381
I20260811 15:11:11.967861 22043 heartbeater.cc:503] Master 127.30.8.62:45969 was elected leader, sending a full tablet report...
Aug 11 15:11:11 dist-test-slave-1vmg krb5kdc[20707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461071, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for kudu/127.30.8.1@KRBTEST.COM
I20260811 15:11:11.975488 22055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.964509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:40023 (local address 127.0.0.1:52600)
0811 15:11:11.965274 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:11.965315 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:11.965457 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:11.965955 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:11.966094 (+ 139us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:11.966484 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:11.966630 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:11.966640 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.968023 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.968026 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:11.968968 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:11.968982 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.969674 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.969692 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:11.969801 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:11.970331 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:11.970343 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:11.971839 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:11.973608 (+ 1769us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:11.973619 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:11.973624 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:11.973926 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:11.974217 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:11.974220 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:11.974222 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:11.974319 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:11.974740 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:11.974745 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:11.974879 (+ 134us) client_negotiation.cc:769] Sending connection context
0811 15:11:11.975055 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 15:11:11.975274 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":602,"thread_start_us":152,"threads_started":1}
I20260811 15:11:11.975500 22057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:11.964583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52600 (local address 127.30.8.1:40023)
0811 15:11:11.965006 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:11.965018 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:11.965572 (+ 554us) server_negotiation.cc:407] Connection header received
0811 15:11:11.965640 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:11.965644 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:11.965809 (+ 165us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:11.965925 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:11.966769 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.967872 (+ 1103us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.969111 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:11.969573 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:11.969674 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:11.971981 (+ 2307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:11.971991 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:11.972003 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:11.972026 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:11.973495 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:11.974032 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:11.974037 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:11.974038 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:11.974129 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:11.974393 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:11.974395 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:11.974396 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:11.974638 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:11.974741 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:11.975161 (+ 420us) server_negotiation.cc:299] Negotiation successful
0811 15:11:11.975364 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":100,"threads_started":1}
I20260811 15:11:11.976783 21924 tablet_service.cc:1514] Processing DeleteTablet for tablet ca1527df3d034a279eaaf3fcd4428d9a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:11:11 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:52600
I20260811 15:11:11.977811 22058 tablet_replica.cc:330] stopping tablet replica
I20260811 15:11:11.978117 22058 raft_consensus.cc:2229] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:11:11.978364 22058 raft_consensus.cc:2258] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:11:11.979403 22058 ts_tablet_manager.cc:1905] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:11:11.983050 22058 ts_tablet_manager.cc:1918] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:11:11.983157 22058 log.cc:1196] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/wal/wals/ca1527df3d034a279eaaf3fcd4428d9a
I20260811 15:11:11.983392 22058 ts_tablet_manager.cc:1939] T ca1527df3d034a279eaaf3fcd4428d9a P eda565b102f448078de510d0a0e548f0: Deleting consensus metadata
I20260811 15:11:11.985589 21692 catalog_manager.cc:4841] TS eda565b102f448078de510d0a0e548f0 (127.30.8.1:40023): tablet ca1527df3d034a279eaaf3fcd4428d9a (table test-table [id=ae5de5e561bf4cfaad930b6ebb5c7638]) successfully deleted
W20260811 15:11:12.065812 22045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:12.066110 22045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:12.066155 22045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:12.070246 22045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:12.070333 22045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:12.070392 22045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:11:12.075928 22045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:41153
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=38361
--tserver_master_addrs=127.30.8.62:45969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:12.077165 22045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:12.078203 22045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:12.088047 22073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:12.088037 22074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:12.088030 22076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:12.088104 22045 server_base.cc:1034] running on GCE node
I20260811 15:11:12.088657 22045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:12.089322 22045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:12.090478 22045 hybrid_clock.cc:648] HybridClock initialized: now 1786461072090464 us; error 37 us; skew 500 ppm
Aug 11 15:11:12 dist-test-slave-1vmg krb5kdc[20707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461072, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:12.093250 22045 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:11:12.094259 22045 webserver.cc:460] Webserver started at http://127.30.8.2:38361/ using document root <none> and password file <none>
I20260811 15:11:12.094880 22045 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:12.095006 22045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:12.099027 22045 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 15:11:12.101169 22087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:12.102392 22045 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 15:11:12.102509 22045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c"
format_stamp: "Formatted at 2026-08-11 15:11:10 on dist-test-slave-1vmg"
server_key: "1c5a5c4d46bdf0e8926c0d702855e20b"
server_key_iv: "b41602f62a71ff5ec820b83cbfc84aa2"
server_key_version: "encryptionkey@0"
I20260811 15:11:12.103106 22045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:12.119498 22045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:12.122424 22045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:12.122628 22045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:12.123284 22045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:11:12.124208 22045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:11:12.124260 22045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:12.124318 22045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:11:12.124347 22045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:12.133206 22045 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:41153
I20260811 15:11:12.133225 22298 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:41153 every 8 connection(s)
I20260811 15:11:12.134268 22045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:11:12.137410 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 22045
I20260811 15:11:12.137838 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:43089
--local_ip_for_outbound_sockets=127.30.8.3
--tserver_master_addrs=127.30.8.62:45969
--webserver_port=37021
--webserver_interface=127.30.8.3
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 15:11:12 dist-test-slave-1vmg krb5kdc[20707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461072, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:12.148346 22046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:12.136172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:45745 (local address 127.30.8.62:45969)
0811 15:11:12.136334 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:12.136339 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:12.137230 (+ 891us) server_negotiation.cc:407] Connection header received
0811 15:11:12.138661 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:12.138665 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:12.138735 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:12.138811 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:12.140083 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.140623 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.141772 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.142008 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.142101 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:12.144823 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:12.144827 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:12.144828 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:12.144859 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:12.146225 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:12.146734 (+ 509us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:12.146737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:12.146739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:12.146820 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:12.147182 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:12.147186 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:12.147188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:12.147382 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:12.147473 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:12.148021 (+ 548us) server_negotiation.cc:299] Negotiation successful
0811 15:11:12.148169 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 15:11:12.149211 22304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:12.136608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:45745)
0811 15:11:12.137070 (+ 462us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:12.137114 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:12.138434 (+ 1320us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:12.139027 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:12.139034 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:12.139405 (+ 371us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:12.139927 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:12.139942 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.140754 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.140757 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:12.141642 (+ 885us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:12.141653 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.142160 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.142164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:12.142341 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:12.142981 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:12.142997 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:12.144636 (+ 1639us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:12.146340 (+ 1704us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:12.146346 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:12.146366 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:12.146627 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:12.146907 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:12.146910 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:12.146938 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:12.147092 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:12.147485 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:12.147493 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:12.147770 (+ 277us) client_negotiation.cc:769] Sending connection context
0811 15:11:12.147945 (+ 175us) client_negotiation.cc:240] Negotiation successful
0811 15:11:12.148315 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":118,"threads_started":1}
I20260811 15:11:12.150413 22299 heartbeater.cc:347] Connected to a master server at 127.30.8.62:45969
I20260811 15:11:12.150691 22299 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:12.151357 22299 heartbeater.cc:511] Master 127.30.8.62:45969 requested a full tablet report, sending...
I20260811 15:11:12.152558 21716 ts_manager.cc:194] Registered new tserver with Master: e393586d25cb4b19a6ffdcc9e9f2a55c (127.30.8.2:41153)
I20260811 15:11:12.153366 21716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:45745
W20260811 15:11:12.258107 22306 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:12.258411 22306 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:12.258463 22306 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:12.262765 22306 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:12.262853 22306 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:12.262955 22306 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:11:12.268551 22306 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:43089
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=37021
--tserver_master_addrs=127.30.8.62:45969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22306
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:12.269744 22306 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:12.270850 22306 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:12.280761 22314 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:12.280774 22313 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:12.280752 22318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:12.281050 22306 server_base.cc:1034] running on GCE node
I20260811 15:11:12.281555 22306 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:12.282272 22306 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:12.283423 22306 hybrid_clock.cc:648] HybridClock initialized: now 1786461072283398 us; error 45 us; skew 500 ppm
Aug 11 15:11:12 dist-test-slave-1vmg krb5kdc[20707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461072, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:12.286448 22306 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:11:12.287420 22306 webserver.cc:460] Webserver started at http://127.30.8.3:37021/ using document root <none> and password file <none>
I20260811 15:11:12.287993 22306 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:12.288089 22306 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:12.292094 22306 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 15:11:12.294379 22330 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:12.295698 22306 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:11:12.295814 22306 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "2ca3edc257ee4a9bb26aa80b674385ba"
format_stamp: "Formatted at 2026-08-11 15:11:11 on dist-test-slave-1vmg"
server_key: "3e658ad93189878ba6e904db79dc7f83"
server_key_iv: "aa9d4215290f107946340210f39ed056"
server_key_version: "encryptionkey@0"
I20260811 15:11:12.296375 22306 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:12.314527 22306 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:12.317919 22306 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:12.318125 22306 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:12.318758 22306 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:11:12.319701 22306 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:11:12.319775 22306 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:12.319833 22306 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:11:12.319865 22306 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:12.328914 22306 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:43089
I20260811 15:11:12.328939 22527 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:43089 every 8 connection(s)
I20260811 15:11:12.329975 22306 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:11:12.335522 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 22306
Aug 11 15:11:12 dist-test-slave-1vmg krb5kdc[20707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461072, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:12.344314 22536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:12.336194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56194 (local address 127.30.8.62:45969)
0811 15:11:12.336417 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:12.336421 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:12.336505 (+ 84us) server_negotiation.cc:407] Connection header received
0811 15:11:12.336624 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:12.336626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:12.336667 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:12.336733 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:12.337608 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.338182 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.339224 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.339455 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.339544 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:12.340936 (+ 1392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:12.340940 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:12.340942 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:12.340965 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:12.342632 (+ 1667us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:12.343146 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:12.343149 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:12.343150 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:12.343231 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:12.343506 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:12.343510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:12.343511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:12.343714 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:12.343940 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:12.344059 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 15:11:12.344167 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":70,"threads_started":1}
I20260811 15:11:12.344986 22046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:12.331916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:59223 (local address 127.30.8.62:45969)
0811 15:11:12.332059 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:12.332062 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:12.332982 (+ 920us) server_negotiation.cc:407] Connection header received
0811 15:11:12.334349 (+ 1367us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:12.334353 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:12.334413 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:12.334490 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:12.335821 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.336462 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.337988 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.338315 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.338386 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:12.341170 (+ 2784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:12.341174 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:12.341176 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:12.341218 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:12.342659 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:12.343244 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:12.343247 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:12.343249 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:12.343325 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:12.343696 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:12.343699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:12.343701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:12.343874 (+ 173us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:12.343956 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:12.344637 (+ 681us) server_negotiation.cc:299] Negotiation successful
0811 15:11:12.344752 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":14,"server-negotiator.queue_time_us":55}
I20260811 15:11:12.345902 22534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:12.332338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:59223)
0811 15:11:12.332827 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:12.332870 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:12.334131 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:12.334698 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:12.334705 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:12.335134 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:12.335669 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:12.335688 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.336603 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.336608 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:12.337883 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:12.337893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.338455 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.338460 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:12.338656 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:12.339292 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:12.339310 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:12.340991 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:12.342769 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:12.342775 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:12.342795 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:12.343165 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:12.343410 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:12.343413 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:12.343414 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:12.343597 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:12.343978 (+ 381us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:12.343985 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:12.344338 (+ 353us) client_negotiation.cc:769] Sending connection context
0811 15:11:12.344566 (+ 228us) client_negotiation.cc:240] Negotiation successful
0811 15:11:12.344929 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":121,"threads_started":1}
I20260811 15:11:12.346978 22531 heartbeater.cc:347] Connected to a master server at 127.30.8.62:45969
I20260811 15:11:12.347229 22531 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:12.347754 22531 heartbeater.cc:511] Master 127.30.8.62:45969 requested a full tablet report, sending...
I20260811 15:11:12.348783 21716 ts_manager.cc:194] Registered new tserver with Master: 2ca3edc257ee4a9bb26aa80b674385ba (127.30.8.3:43089)
I20260811 15:11:12.349335 21716 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:59223
I20260811 15:11:12.357915 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 15:11:12.362447 22046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:12.358553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56196 (local address 127.30.8.62:45969)
0811 15:11:12.358677 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:12.358682 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:12.358929 (+ 247us) server_negotiation.cc:407] Connection header received
0811 15:11:12.359121 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:12.359124 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:12.359243 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:12.359347 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:11:12.359590 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.360121 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.360983 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.361201 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.361291 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:12.361609 (+ 318us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:11:12.362062 (+ 453us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:11:12.362174 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:12.362241 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 15:11:12.362300 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 15:11:12.363232 21716 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:56196:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:11:12.366549 21716 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:11:12.374495 21924 tablet_service.cc:1467] Processing CreateTablet for tablet cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e (DEFAULT_TABLE table=test-table [id=99dc2caff9034105a7f8281cd1927a1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:11:12.375306 21924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:12.378327 21851 tablet_bootstrap.cc:492] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0: Bootstrap starting.
I20260811 15:11:12.378773 22055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:12.373290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:41153 (local address 127.0.0.1:47564)
0811 15:11:12.373453 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:12.373468 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:12.373559 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:12.374595 (+ 1036us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:12.374598 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:12.374617 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:12.374732 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:12.374739 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.375956 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.375960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:12.377189 (+ 1229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:12.377200 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.378338 (+ 1138us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.378342 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:12.378445 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:12.378500 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 15:11:12.378554 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 15:11:12.378644 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 15:11:12.378985 22545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:12.373498 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:43089 (local address 127.0.0.1:57168)
0811 15:11:12.373783 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:12.373795 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:12.373884 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:12.374678 (+ 794us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:12.374682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:12.374703 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:12.374802 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:12.374808 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.376808 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.376812 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:12.377684 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:12.377694 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.378572 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.378576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:12.378672 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:12.378699 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 15:11:12.378750 (+ 51us) client_negotiation.cc:240] Negotiation successful
0811 15:11:12.378824 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":126,"threads_started":1}
I20260811 15:11:12.379364 21851 tablet_bootstrap.cc:654] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0: Neither blocks nor log segments found. Creating new log.
I20260811 15:11:12.379612 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:12.373741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57168 (local address 127.30.8.3:43089)
0811 15:11:12.374205 (+ 464us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:12.374217 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:12.374242 (+ 25us) server_negotiation.cc:407] Connection header received
0811 15:11:12.374319 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:12.374324 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:12.374516 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:12.375024 (+ 508us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:12.375205 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.376660 (+ 1455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.377812 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.378430 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.379161 (+ 731us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:12.379313 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:12.379368 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 15:11:12.379468 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":354,"thread_start_us":153,"threads_started":1}
I20260811 15:11:12.379680 22547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:12.373502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47564 (local address 127.30.8.2:41153)
0811 15:11:12.374133 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:12.374146 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:12.374173 (+ 27us) server_negotiation.cc:407] Connection header received
0811 15:11:12.374246 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:12.374251 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:12.374429 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:12.374591 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:12.374870 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.375804 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.377336 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.378197 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.379095 (+ 898us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:12.379267 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:12.379323 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 15:11:12.379423 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":521,"thread_start_us":126,"threads_started":1}
I20260811 15:11:12.382709 21851 tablet_bootstrap.cc:492] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0: No bootstrap required, opened a new log
I20260811 15:11:12.382804 21851 ts_tablet_manager.cc:1397] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 15:11:12.382550 22184 tablet_service.cc:1467] Processing CreateTablet for tablet cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e (DEFAULT_TABLE table=test-table [id=99dc2caff9034105a7f8281cd1927a1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:11:12.382545 22409 tablet_service.cc:1467] Processing CreateTablet for tablet cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e (DEFAULT_TABLE table=test-table [id=99dc2caff9034105a7f8281cd1927a1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:11:12.383327 21851 raft_consensus.cc:348] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } } peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } }
I20260811 15:11:12.383461 21851 raft_consensus.cc:374] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:12.383512 21851 raft_consensus.cc:729] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda565b102f448078de510d0a0e548f0, State: Initialized, Role: FOLLOWER
I20260811 15:11:12.383680 21851 consensus_queue.cc:260] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } } peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } }
I20260811 15:11:12.383777 22184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:12.383780 22409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:12.383962 21851 ts_tablet_manager.cc:1428] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 15:11:12.391474 22553 tablet_bootstrap.cc:492] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c: Bootstrap starting.
I20260811 15:11:12.392242 22556 tablet_bootstrap.cc:492] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: Bootstrap starting.
I20260811 15:11:12.393867 22556 tablet_bootstrap.cc:654] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: Neither blocks nor log segments found. Creating new log.
I20260811 15:11:12.394109 22553 tablet_bootstrap.cc:654] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c: Neither blocks nor log segments found. Creating new log.
I20260811 15:11:12.394060 22054 raft_consensus.cc:482] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:11:12.394263 22054 raft_consensus.cc:504] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } } peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } }
I20260811 15:11:12.395010 22556 log.cc:824] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:12.395038 22553 log.cc:824] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:12.395673 22054 leader_election.cc:288] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e393586d25cb4b19a6ffdcc9e9f2a55c (127.30.8.2:41153), 2ca3edc257ee4a9bb26aa80b674385ba (127.30.8.3:43089)
I20260811 15:11:12.397426 22556 tablet_bootstrap.cc:492] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: No bootstrap required, opened a new log
I20260811 15:11:12.397572 22553 tablet_bootstrap.cc:492] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c: No bootstrap required, opened a new log
I20260811 15:11:12.397640 22556 ts_tablet_manager.cc:1397] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260811 15:11:12.397763 22553 ts_tablet_manager.cc:1397] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 15:11:12.401330 22549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:12.396285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:58087 (local address 127.30.8.3:43089)
0811 15:11:12.396385 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:12.396388 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:12.396400 (+ 12us) server_negotiation.cc:407] Connection header received
0811 15:11:12.396442 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:12.396444 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:12.396479 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:12.396541 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:12.396873 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.397565 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.399807 (+ 2242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.400413 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.400519 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:12.400595 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:12.401116 (+ 521us) server_negotiation.cc:299] Negotiation successful
0811 15:11:12.401202 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260811 15:11:12.401710 22561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:12.395926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:43089 (local address 127.30.8.1:58087)
0811 15:11:12.396212 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:12.396227 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:12.396332 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:12.396568 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:12.396570 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:12.396595 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:12.396727 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:12.396734 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.398075 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.398078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:12.399617 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:12.399631 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.400835 (+ 1204us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.400839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:12.400942 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:12.400998 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 15:11:12.401463 (+ 465us) client_negotiation.cc:240] Negotiation successful
0811 15:11:12.401550 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":90,"threads_started":1}
I20260811 15:11:12.401611 22553 raft_consensus.cc:348] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } } peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } }
I20260811 15:11:12.401693 22556 raft_consensus.cc:348] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } } peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } }
I20260811 15:11:12.401918 22553 raft_consensus.cc:374] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:12.401952 22556 raft_consensus.cc:374] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:12.401976 22553 raft_consensus.cc:729] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e393586d25cb4b19a6ffdcc9e9f2a55c, State: Initialized, Role: FOLLOWER
I20260811 15:11:12.402000 22556 raft_consensus.cc:729] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ca3edc257ee4a9bb26aa80b674385ba, State: Initialized, Role: FOLLOWER
I20260811 15:11:12.402343 22452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e" candidate_uuid: "eda565b102f448078de510d0a0e548f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" is_pre_election: true
I20260811 15:11:12.402493 22556 consensus_queue.cc:260] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } } peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } }
I20260811 15:11:12.402493 22553 consensus_queue.cc:260] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } } peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } }
I20260811 15:11:12.402745 22452 raft_consensus.cc:2454] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eda565b102f448078de510d0a0e548f0 in term 0.
I20260811 15:11:12.403265 21843 leader_election.cc:302] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ca3edc257ee4a9bb26aa80b674385ba, eda565b102f448078de510d0a0e548f0; no voters:
I20260811 15:11:12.403609 22049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:12.395872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:41153 (local address 127.30.8.1:52567)
0811 15:11:12.396065 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:12.396080 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:12.396181 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:12.399356 (+ 3175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:12.399360 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:12.399400 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:12.399496 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:12.399513 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.400540 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.400544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:12.401647 (+ 1103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:12.401659 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.403196 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.403200 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:12.403298 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:12.403321 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 15:11:12.403395 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 15:11:12.403475 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104}
I20260811 15:11:12.403610 22556 ts_tablet_manager.cc:1428] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 15:11:12.403610 22553 ts_tablet_manager.cc:1428] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260811 15:11:12.403568 22054 raft_consensus.cc:2781] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:11:12.403857 22054 raft_consensus.cc:482] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:11:12.403923 22054 raft_consensus.cc:3037] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:12.403972 22531 heartbeater.cc:503] Master 127.30.8.62:45969 was elected leader, sending a full tablet report...
I20260811 15:11:12.404078 22299 heartbeater.cc:503] Master 127.30.8.62:45969 was elected leader, sending a full tablet report...
I20260811 15:11:12.404438 22547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:12.395976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:52567 (local address 127.30.8.2:41153)
0811 15:11:12.399052 (+ 3076us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:12.399056 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:12.399072 (+ 16us) server_negotiation.cc:407] Connection header received
0811 15:11:12.399120 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:12.399122 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:12.399166 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:12.399241 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:12.399663 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.400396 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.402214 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.402809 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.402909 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:12.402997 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:12.404254 (+ 1257us) server_negotiation.cc:299] Negotiation successful
0811 15:11:12.404333 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2993}
I20260811 15:11:12.404971 22218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e" candidate_uuid: "eda565b102f448078de510d0a0e548f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" is_pre_election: true
I20260811 15:11:12.405238 22218 raft_consensus.cc:2454] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eda565b102f448078de510d0a0e548f0 in term 0.
I20260811 15:11:12.405329 22054 raft_consensus.cc:504] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } } peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } }
I20260811 15:11:12.405707 22054 leader_election.cc:288] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [CANDIDATE]: Term 1 election: Requested vote from peers e393586d25cb4b19a6ffdcc9e9f2a55c (127.30.8.2:41153), 2ca3edc257ee4a9bb26aa80b674385ba (127.30.8.3:43089)
I20260811 15:11:12.406049 22452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e" candidate_uuid: "eda565b102f448078de510d0a0e548f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ca3edc257ee4a9bb26aa80b674385ba"
I20260811 15:11:12.406137 22218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e" candidate_uuid: "eda565b102f448078de510d0a0e548f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c"
I20260811 15:11:12.406176 22452 raft_consensus.cc:3037] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:12.406237 22218 raft_consensus.cc:3037] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:12.407509 22452 raft_consensus.cc:2454] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eda565b102f448078de510d0a0e548f0 in term 1.
I20260811 15:11:12.407519 22218 raft_consensus.cc:2454] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eda565b102f448078de510d0a0e548f0 in term 1.
I20260811 15:11:12.407848 21843 leader_election.cc:302] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 2ca3edc257ee4a9bb26aa80b674385ba, e393586d25cb4b19a6ffdcc9e9f2a55c, eda565b102f448078de510d0a0e548f0; no voters:
I20260811 15:11:12.408071 22054 raft_consensus.cc:2781] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:11:12.408365 22054 raft_consensus.cc:686] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 1 LEADER]: Becoming Leader. State: Replica: eda565b102f448078de510d0a0e548f0, State: Running, Role: LEADER
I20260811 15:11:12.408679 22054 consensus_queue.cc:237] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } } peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } }
I20260811 15:11:12.411640 21716 catalog_manager.cc:5495] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 reported cstate change: term changed from 0 to 1, leader changed from <none> to eda565b102f448078de510d0a0e548f0 (127.30.8.1). New cstate: current_term: 1 leader_uuid: "eda565b102f448078de510d0a0e548f0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } health_report { overall_health: HEALTHY } } }
I20260811 15:11:12.423635 22057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:12.419958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52616 (local address 127.30.8.1:40023)
0811 15:11:12.420119 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:12.420125 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:12.420141 (+ 16us) server_negotiation.cc:407] Connection header received
0811 15:11:12.420288 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:12.420291 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:12.420361 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:12.420463 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:11:12.420768 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.421315 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.422115 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:12.422363 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:12.422454 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:12.422720 (+ 266us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:11:12.423242 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:11:12.423350 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:12.423425 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 15:11:12.423487 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 15:11:12.427662 22218 raft_consensus.cc:1261] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c [term 1 FOLLOWER]: Refusing update from remote peer eda565b102f448078de510d0a0e548f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:11:12.427662 22452 raft_consensus.cc:1261] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba [term 1 FOLLOWER]: Refusing update from remote peer eda565b102f448078de510d0a0e548f0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:11:12.428421 22567 consensus_queue.cc:1035] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:11:12.428612 22054 consensus_queue.cc:1035] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:11:12.435513 22572 mvcc.cc:205] Tried to move back new op lower bound from 7317344552657289216 to 7317344552588845056. Current Snapshot: MvccSnapshot[applied={T|T < 7317344552657289216}]
I20260811 15:11:12.439303 22576 mvcc.cc:205] Tried to move back new op lower bound from 7317344552657289216 to 7317344552588845056. Current Snapshot: MvccSnapshot[applied={T|T < 7317344552657289216}]
I20260811 15:11:12.444471 21716 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:56196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:11:12.444641 21716 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:56196:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:11:12.446978 21716 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97: Sending DeleteTablet for 3 replicas of tablet cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e
I20260811 15:11:12.447734 21924 tablet_service.cc:1514] Processing DeleteTablet for tablet cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:11:12 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:52600
I20260811 15:11:12.447785 22409 tablet_service.cc:1514] Processing DeleteTablet for tablet cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:11:12 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:57168
I20260811 15:11:12.447760 22184 tablet_service.cc:1514] Processing DeleteTablet for tablet cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:11:12 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:47564
I20260811 15:11:12.447952 22058 tablet_replica.cc:330] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0: stopping tablet replica
I20260811 15:11:12.448089 22058 raft_consensus.cc:2229] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:11:12.448312 22592 tablet_replica.cc:330] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: stopping tablet replica
I20260811 15:11:12.448405 22058 raft_consensus.cc:2258] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:11:12.448511 22593 tablet_replica.cc:330] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c: stopping tablet replica
I20260811 15:11:12.448549 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 21781
I20260811 15:11:12.457333 22592 raft_consensus.cc:2229] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:11:12.457836 22592 raft_consensus.cc:2258] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:11:12.459482 22592 ts_tablet_manager.cc:1905] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:11:12.461792 22592 ts_tablet_manager.cc:1918] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:11:12.461899 22592 log.cc:1196] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e
I20260811 15:11:12.462219 22592 ts_tablet_manager.cc:1939] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: Deleting consensus metadata
I20260811 15:11:12.462224 22593 raft_consensus.cc:2229] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:11:12.462671 22593 raft_consensus.cc:2258] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:11:12.463699 21692 catalog_manager.cc:4841] TS 2ca3edc257ee4a9bb26aa80b674385ba (127.30.8.3:43089): tablet cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e (table test-table [id=99dc2caff9034105a7f8281cd1927a1c]) successfully deleted
I20260811 15:11:12.464244 22593 ts_tablet_manager.cc:1905] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:11:12.465384 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 22045
W20260811 15:11:12.465442 21692 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.8.1:40023 (error 108)
I20260811 15:11:12.473412 22545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:12.473044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:40023 (local address 127.0.0.1:52624)
0811 15:11:12.473249 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:12.473316 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.1:40023: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
W20260811 15:11:12.473613 21692 catalog_manager.cc:4584] TS eda565b102f448078de510d0a0e548f0 (127.30.8.1:40023): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e: Network error: Client connection negotiation failed: client connection to 127.30.8.1:40023: connect: Connection refused (error 111)
I20260811 15:11:12.483212 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 22306
I20260811 15:11:12.488044 22545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:12.483365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:41153 (local address 127.0.0.1:47576)
0811 15:11:12.487832 (+ 4467us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:12.487923 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.2:41153: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4331}
I20260811 15:11:12.499380 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 21654
I20260811 15:11:12.519079 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:45969
--webserver_interface=127.30.8.62
--webserver_port=43883
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:45969
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:11:12.636370 22595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:12.636683 22595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:12.636727 22595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:12.640946 22595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:11:12.641031 22595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:12.641050 22595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:12.641067 22595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:11:12.641080 22595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:11:12.646811 22595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:45969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:45969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=43883
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:12.648030 22595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:12.649115 22595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:12.658202 22606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:12.658202 22603 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:12.658207 22602 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:12.658496 22595 server_base.cc:1034] running on GCE node
I20260811 15:11:12.659186 22595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:12.660391 22595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:12.661623 22595 hybrid_clock.cc:648] HybridClock initialized: now 1786461072661596 us; error 70 us; skew 500 ppm
Aug 11 15:11:12 dist-test-slave-1vmg krb5kdc[20707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461072, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:12.664669 22595 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:11:12.665666 22595 webserver.cc:460] Webserver started at http://127.30.8.62:43883/ using document root <none> and password file <none>
I20260811 15:11:12.666271 22595 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:12.666337 22595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:12.670372 22595 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:11:12.672811 22619 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:12.674064 22595 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:11:12.674196 22595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "50c523135e324a059998f8c92d341b97"
format_stamp: "Formatted at 2026-08-11 15:11:10 on dist-test-slave-1vmg"
server_key: "d6fd09097442b44787fb9a5d76c9096d"
server_key_iv: "af884792ead4d1ed38f96cd65cdd9297"
server_key_version: "encryptionkey@0"
I20260811 15:11:12.674778 22595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:12.697614 22595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:12.700870 22595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:12.701069 22595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:12.708719 22595 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:45969
I20260811 15:11:12.709051 22714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:45969 every 8 connection(s)
I20260811 15:11:12.709887 22595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:11:12.714490 22716 sys_catalog.cc:263] Verifying existing consensus state
I20260811 15:11:12.716331 22716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97: Bootstrap starting.
I20260811 15:11:12.716923 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 22595
I20260811 15:11:12.717463 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:40023
--local_ip_for_outbound_sockets=127.30.8.1
--tserver_master_addrs=127.30.8.62:45969
--webserver_port=40379
--webserver_interface=127.30.8.1
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:11:12.734241 22716 log.cc:824] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:12.745873 22716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 15:11:12.746297 22716 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97: Bootstrap complete.
I20260811 15:11:12.750896 22716 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c523135e324a059998f8c92d341b97" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45969 } }
I20260811 15:11:12.751364 22716 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50c523135e324a059998f8c92d341b97, State: Initialized, Role: FOLLOWER
I20260811 15:11:12.751899 22716 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c523135e324a059998f8c92d341b97" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45969 } }
I20260811 15:11:12.752034 22716 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:11:12.752104 22716 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:11:12.752197 22716 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 2 FOLLOWER]: Advancing to term 3
I20260811 15:11:12.754220 22716 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c523135e324a059998f8c92d341b97" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45969 } }
I20260811 15:11:12.754550 22716 leader_election.cc:302] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50c523135e324a059998f8c92d341b97; no voters:
I20260811 15:11:12.754871 22716 leader_election.cc:288] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 15:11:12.755051 22721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 15:11:12.755317 22721 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [term 3 LEADER]: Becoming Leader. State: Replica: 50c523135e324a059998f8c92d341b97, State: Running, Role: LEADER
I20260811 15:11:12.755662 22721 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c523135e324a059998f8c92d341b97" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45969 } }
I20260811 15:11:12.756142 22716 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:11:12.757292 22725 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50c523135e324a059998f8c92d341b97. Latest consensus state: current_term: 3 leader_uuid: "50c523135e324a059998f8c92d341b97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c523135e324a059998f8c92d341b97" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45969 } } }
I20260811 15:11:12.757426 22725 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [sys.catalog]: This master's current role is: LEADER
I20260811 15:11:12.757740 22723 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "50c523135e324a059998f8c92d341b97" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c523135e324a059998f8c92d341b97" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45969 } } }
I20260811 15:11:12.757834 22723 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50c523135e324a059998f8c92d341b97 [sys.catalog]: This master's current role is: LEADER
I20260811 15:11:12.757990 22741 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:11:12.763458 22741 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:11:12.764249 22741 catalog_manager.cc:1259] Loaded cluster ID: 0effabe709ff474ca096d5222d750fed
I20260811 15:11:12.764369 22741 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 15:11:12.765555 22741 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f8d354439ad google::LogMessage::Fail() at ??:0
@ 0x7f8d3544688a google::LogMessage::SendToLog() at ??:0
@ 0x7f8d354434de google::LogMessage::Flush() at ??:0
@ 0x7f8d35444a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f8d3fd82fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f8d3fd83d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f8d3fd80579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f8d3fdc4640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f8d376953d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f8d37695c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f8d376973f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f8d37686512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f8d38cca6db start_thread at ??:0
@ 0x7f8d33f4161f clone at ??:0
W20260811 15:11:12.866993 22718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:12.867412 22718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:12.867492 22718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:12.873597 22718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:12.873710 22718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:12.873816 22718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:11:12.882078 22718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:40023
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=40379
--tserver_master_addrs=127.30.8.62:45969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:12.883751 22718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:12.885219 22718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:12.896975 22783 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:12.896957 22780 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:12.896941 22781 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:12.897445 22718 server_base.cc:1034] running on GCE node
I20260811 15:11:12.897984 22718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:12.898862 22718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:12.900089 22718 hybrid_clock.cc:648] HybridClock initialized: now 1786461072900058 us; error 40 us; skew 500 ppm
Aug 11 15:11:12 dist-test-slave-1vmg krb5kdc[20707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461072, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:12.903702 22718 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:11:12.904979 22718 webserver.cc:460] Webserver started at http://127.30.8.1:40379/ using document root <none> and password file <none>
I20260811 15:11:12.905813 22718 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:12.905915 22718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:12.911584 22718 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 15:11:12.914615 22793 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:12.916272 22718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:11:12.916420 22718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "eda565b102f448078de510d0a0e548f0"
format_stamp: "Formatted at 2026-08-11 15:11:10 on dist-test-slave-1vmg"
server_key: "4fca43a9641337d447dfa02283fb492c"
server_key_iv: "5ef529ee9df4a0fb1e2760f42861647d"
server_key_version: "encryptionkey@0"
I20260811 15:11:12.917172 22718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:12.944202 22718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:12.948426 22718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:12.948676 22718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:12.949528 22718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:11:12.951078 22803 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 15:11:12.955559 22718 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 15:11:12.955662 22718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.001s
I20260811 15:11:12.955754 22718 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 15:11:12.958392 22718 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 15:11:12.958472 22718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260811 15:11:12.958756 22803 tablet_bootstrap.cc:492] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0: Bootstrap starting.
I20260811 15:11:12.969710 22718 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:40023
I20260811 15:11:12.970098 22988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:40023 every 8 connection(s)
I20260811 15:11:12.971118 22718 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:11:12.977604 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 22718
I20260811 15:11:12.978135 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:41153
--local_ip_for_outbound_sockets=127.30.8.2
--tserver_master_addrs=127.30.8.62:45969
--webserver_port=38361
--webserver_interface=127.30.8.2
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:11:12.978602 22803 log.cc:824] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:12.987488 22803 tablet_bootstrap.cc:492] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 15:11:12.987978 22803 tablet_bootstrap.cc:492] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0: Bootstrap complete.
I20260811 15:11:12.989352 22803 ts_tablet_manager.cc:1397] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0: Time spent bootstrapping tablet: real 0.031s user 0.016s sys 0.012s
I20260811 15:11:12.992914 22803 raft_consensus.cc:348] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } } peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } }
I20260811 15:11:12.993507 22803 raft_consensus.cc:729] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eda565b102f448078de510d0a0e548f0, State: Initialized, Role: FOLLOWER
I20260811 15:11:12.994087 22803 consensus_queue.cc:260] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } } peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } }
I20260811 15:11:12.995220 22803 ts_tablet_manager.cc:1428] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260811 15:11:13.053764 22992 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:11:12.973641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:33569)
0811 15:11:12.974269 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:12.974331 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:12.975472 (+ 1141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:13.052748 (+ 77276us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":163,"threads_started":1}
W20260811 15:11:13.054423 22990 heartbeater.cc:644] Failed to heartbeat to 127.30.8.62:45969 (0 consecutive failures): Network error: Failed to ping master at 127.30.8.62:45969: Client connection negotiation failed: client connection to 127.30.8.62:45969: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 15:11:13.055292 22992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:13.054989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:57325)
0811 15:11:13.055131 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:13.055217 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 15:11:13.056015 22992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:13.055804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:42617)
0811 15:11:13.055907 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:13.055957 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260811 15:11:13.056212 22990 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 15:11:13.103106 22998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:13.103415 22998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:13.103478 22998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:13.107676 22998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:13.107752 22998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:13.107841 22998 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:11:13.113461 22998 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:41153
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=38361
--tserver_master_addrs=127.30.8.62:45969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:13.114673 22998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:13.115764 22998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:13.125666 23012 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:13.125748 23011 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:13.125748 23016 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:13.125908 22998 server_base.cc:1034] running on GCE node
I20260811 15:11:13.126413 22998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:13.127146 22998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:13.128334 22998 hybrid_clock.cc:648] HybridClock initialized: now 1786461073128306 us; error 42 us; skew 500 ppm
Aug 11 15:11:13 dist-test-slave-1vmg krb5kdc[20707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461073, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:13.131116 22998 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:11:13.132052 22998 webserver.cc:460] Webserver started at http://127.30.8.2:38361/ using document root <none> and password file <none>
I20260811 15:11:13.132637 22998 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:13.132707 22998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:13.136740 22998 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 15:11:13.139112 23031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:13.140378 22998 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:11:13.140499 22998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c"
format_stamp: "Formatted at 2026-08-11 15:11:10 on dist-test-slave-1vmg"
server_key: "1c5a5c4d46bdf0e8926c0d702855e20b"
server_key_iv: "b41602f62a71ff5ec820b83cbfc84aa2"
server_key_version: "encryptionkey@0"
I20260811 15:11:13.141067 22998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:13.155864 22998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:13.158875 22998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:13.159134 22998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:13.159794 22998 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:11:13.160928 23040 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 15:11:13.165300 22998 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 15:11:13.165375 22998 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 15:11:13.165446 22998 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 15:11:13.167294 22998 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 15:11:13.167346 22998 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260811 15:11:13.167600 23040 tablet_bootstrap.cc:492] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c: Bootstrap starting.
I20260811 15:11:13.173671 23040 log.cc:824] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:13.177413 22998 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:41153
I20260811 15:11:13.177467 23203 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:41153 every 8 connection(s)
I20260811 15:11:13.178304 22998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:11:13.179239 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 22998
I20260811 15:11:13.179612 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:43089
--local_ip_for_outbound_sockets=127.30.8.3
--tserver_master_addrs=127.30.8.62:45969
--webserver_port=37021
--webserver_interface=127.30.8.3
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:35941
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:11:13.182763 23209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:13.181032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:53271)
0811 15:11:13.181767 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:13.181901 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":148,"threads_started":1}
W20260811 15:11:13.183221 23204 heartbeater.cc:644] Failed to heartbeat to 127.30.8.62:45969 (0 consecutive failures): Network error: Failed to ping master at 127.30.8.62:45969: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
I20260811 15:11:13.183743 23040 tablet_bootstrap.cc:492] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 15:11:13.183933 23209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:13.183684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:39103)
0811 15:11:13.183816 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:13.183870 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 15:11:13.184161 23040 tablet_bootstrap.cc:492] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c: Bootstrap complete.
I20260811 15:11:13.184634 23040 ts_tablet_manager.cc:1397] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c: Time spent bootstrapping tablet: real 0.017s user 0.012s sys 0.004s
I20260811 15:11:13.184998 23209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:13.184774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:54893)
0811 15:11:13.184886 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:13.184938 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 15:11:13.185235 23204 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 15:11:13.187855 23040 raft_consensus.cc:348] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } } peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } }
I20260811 15:11:13.188283 23040 raft_consensus.cc:729] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e393586d25cb4b19a6ffdcc9e9f2a55c, State: Initialized, Role: FOLLOWER
I20260811 15:11:13.188733 23040 consensus_queue.cc:260] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } } peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } }
I20260811 15:11:13.189664 23040 ts_tablet_manager.cc:1428] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260811 15:11:13.189976 23209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:13.189695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:38667)
0811 15:11:13.189847 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:13.189908 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 15:11:13.190572 23209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:13.190398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:36745)
0811 15:11:13.190484 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:13.190521 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 15:11:13.298169 23208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:13.298465 23208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:13.298508 23208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:13.302657 23208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:13.302747 23208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:13.302819 23208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:11:13.308447 23208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:35941
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:43089
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=37021
--tserver_master_addrs=127.30.8.62:45969
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:13.309635 23208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:13.310693 23208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:13.320602 23218 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:13.320601 23220 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:13.320801 23208 server_base.cc:1034] running on GCE node
W20260811 15:11:13.320637 23217 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:13.321317 23208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:13.322014 23208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:13.323175 23208 hybrid_clock.cc:648] HybridClock initialized: now 1786461073323147 us; error 41 us; skew 500 ppm
Aug 11 15:11:13 dist-test-slave-1vmg krb5kdc[20707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461073, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:13.326198 23208 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:11:13.327453 23208 webserver.cc:460] Webserver started at http://127.30.8.3:37021/ using document root <none> and password file <none>
I20260811 15:11:13.328225 23208 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:13.328323 23208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:13.333340 23208 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 15:11:13.335569 23228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:13.336831 23208 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:11:13.336936 23208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "2ca3edc257ee4a9bb26aa80b674385ba"
format_stamp: "Formatted at 2026-08-11 15:11:11 on dist-test-slave-1vmg"
server_key: "3e658ad93189878ba6e904db79dc7f83"
server_key_iv: "aa9d4215290f107946340210f39ed056"
server_key_version: "encryptionkey@0"
I20260811 15:11:13.337474 23208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:13.361598 23208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:13.365015 23208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:13.365193 23208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:13.365782 23208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:11:13.366654 23208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:11:13.366704 23208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:13.366744 23208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:11:13.366765 23208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:13.375665 23208 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:43089
I20260811 15:11:13.375685 23414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:43089 every 8 connection(s)
I20260811 15:11:13.376748 23208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:11:13.378108 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 23208
I20260811 15:11:13.380810 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:13.379143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:39629)
0811 15:11:13.379635 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:13.380077 (+ 442us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":107,"threads_started":1}
W20260811 15:11:13.381253 23415 heartbeater.cc:644] Failed to heartbeat to 127.30.8.62:45969 (0 consecutive failures): Network error: Failed to ping master at 127.30.8.62:45969: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
I20260811 15:11:13.381839 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:13.381641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:55329)
0811 15:11:13.381739 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:13.381783 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260811 15:11:13.382346 23421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:13.382182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:58265)
0811 15:11:13.382262 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:13.382299 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 15:11:13.382483 23415 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 15:11:14.057395 23423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:14.056913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:34479)
0811 15:11:14.057205 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:14.057271 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":88,"threads_started":1}
I20260811 15:11:14.155763 23425 raft_consensus.cc:482] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:11:14.156006 23425 raft_consensus.cc:504] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } } peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } }
I20260811 15:11:14.157155 23425 leader_election.cc:288] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers e393586d25cb4b19a6ffdcc9e9f2a55c (127.30.8.2:41153), 2ca3edc257ee4a9bb26aa80b674385ba (127.30.8.3:43089)
Aug 11 15:11:14 dist-test-slave-1vmg krb5kdc[20707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461072, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.2@KRBTEST.COM
Aug 11 15:11:14 dist-test-slave-1vmg krb5kdc[20707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461072, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.3@KRBTEST.COM
I20260811 15:11:14.170959 23423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:14.157453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:41153 (local address 127.30.8.1:36663)
0811 15:11:14.157624 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:14.157639 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:14.157755 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:14.159043 (+ 1288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:14.159049 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:14.159572 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:14.160202 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:14.160224 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:14.161490 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:14.161493 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:14.162491 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:14.162501 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:14.163242 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:14.163246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:14.163498 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:14.164166 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:14.164178 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:14.165896 (+ 1718us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:14.168796 (+ 2900us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:14.168799 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:14.168802 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:14.168983 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:14.169301 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:14.169304 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:14.169306 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:14.169380 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:14.169963 (+ 583us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:14.169970 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:14.170232 (+ 262us) client_negotiation.cc:769] Sending connection context
0811 15:11:14.170404 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 15:11:14.170742 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":206,"spinlock_wait_cycles":24576}
I20260811 15:11:14.170982 23426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:14.157597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:36663 (local address 127.30.8.2:41153)
0811 15:11:14.157914 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:14.157936 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:14.157986 (+ 50us) server_negotiation.cc:407] Connection header received
0811 15:11:14.158271 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:14.158276 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:14.158641 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:14.158907 (+ 266us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:14.160375 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:14.161345 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:14.162619 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:14.163122 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:14.163284 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:14.166110 (+ 2826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:14.166116 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:14.166130 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:14.166162 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:14.168694 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:14.169090 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:14.169095 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:14.169099 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:14.169209 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:14.169462 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:14.169465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:14.169467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:14.169850 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:14.169965 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:14.170535 (+ 570us) server_negotiation.cc:299] Negotiation successful
0811 15:11:14.170817 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":91,"threads_started":1}
I20260811 15:11:14.171324 23428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:14.157453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:43089 (local address 127.30.8.1:33365)
0811 15:11:14.158152 (+ 699us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:14.158162 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:14.158263 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:14.159623 (+ 1360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:14.159626 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:14.159840 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:14.160200 (+ 360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:14.160226 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:14.161491 (+ 1265us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:14.161495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:14.162486 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:14.162497 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:14.163242 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:14.163245 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:14.163486 (+ 241us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:14.164120 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:14.164137 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:14.166222 (+ 2085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:14.168507 (+ 2285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:14.168513 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:14.168525 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:14.168786 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:14.169162 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:14.169164 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:14.169165 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:14.169302 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:14.169963 (+ 661us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:14.169973 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:14.170228 (+ 255us) client_negotiation.cc:769] Sending connection context
0811 15:11:14.170404 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 15:11:14.171217 (+ 813us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"mutex_wait_us":1090,"spinlock_wait_cycles":12160,"thread_start_us":95,"threads_started":1}
I20260811 15:11:14.171552 23427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:14.157613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:33365 (local address 127.30.8.3:43089)
0811 15:11:14.157981 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:14.157991 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:14.158269 (+ 278us) server_negotiation.cc:407] Connection header received
0811 15:11:14.158981 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:14.158988 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:14.159372 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:14.159658 (+ 286us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:14.160392 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:14.161348 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:14.162608 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:14.163121 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:14.163279 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:14.166431 (+ 3152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:14.166438 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:14.166452 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:14.166486 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:14.168385 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:14.168910 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:14.168917 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:14.168921 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:14.169056 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:14.169396 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:14.169399 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:14.169400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:14.169842 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:14.169965 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:14.170736 (+ 771us) server_negotiation.cc:299] Negotiation successful
0811 15:11:14.171036 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":275,"thread_start_us":94,"threads_started":1}
I20260811 15:11:14.173007 23132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e" candidate_uuid: "eda565b102f448078de510d0a0e548f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" is_pre_election: true
I20260811 15:11:14.173379 23132 raft_consensus.cc:2454] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eda565b102f448078de510d0a0e548f0 in term 1.
W20260811 15:11:14.174288 22796 leader_election.cc:341] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2ca3edc257ee4a9bb26aa80b674385ba (127.30.8.3:43089): Not found: Tablet not found: cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e
I20260811 15:11:14.174407 22796 leader_election.cc:302] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e393586d25cb4b19a6ffdcc9e9f2a55c, eda565b102f448078de510d0a0e548f0; no voters: 2ca3edc257ee4a9bb26aa80b674385ba
I20260811 15:11:14.174676 23425 raft_consensus.cc:2781] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 15:11:14.174759 23425 raft_consensus.cc:482] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:11:14.174790 23425 raft_consensus.cc:3037] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 1 FOLLOWER]: Advancing to term 2
I20260811 15:11:14.176954 23425 raft_consensus.cc:504] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } } peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } }
I20260811 15:11:14.177357 23425 leader_election.cc:288] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [CANDIDATE]: Term 2 election: Requested vote from peers e393586d25cb4b19a6ffdcc9e9f2a55c (127.30.8.2:41153), 2ca3edc257ee4a9bb26aa80b674385ba (127.30.8.3:43089)
I20260811 15:11:14.177772 23132 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e" candidate_uuid: "eda565b102f448078de510d0a0e548f0" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c"
I20260811 15:11:14.177924 23132 raft_consensus.cc:3037] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c [term 1 FOLLOWER]: Advancing to term 2
W20260811 15:11:14.178107 22796 leader_election.cc:341] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 2ca3edc257ee4a9bb26aa80b674385ba (127.30.8.3:43089): Not found: Tablet not found: cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e
I20260811 15:11:14.180263 23132 raft_consensus.cc:2454] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eda565b102f448078de510d0a0e548f0 in term 2.
I20260811 15:11:14.180594 22798 leader_election.cc:302] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: e393586d25cb4b19a6ffdcc9e9f2a55c, eda565b102f448078de510d0a0e548f0; no voters: 2ca3edc257ee4a9bb26aa80b674385ba
I20260811 15:11:14.180774 23425 raft_consensus.cc:2781] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 15:11:14.180981 23425 raft_consensus.cc:686] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [term 2 LEADER]: Becoming Leader. State: Replica: eda565b102f448078de510d0a0e548f0, State: Running, Role: LEADER
I20260811 15:11:14.181342 23425 consensus_queue.cc:237] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } } peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } }
I20260811 15:11:14.186297 23428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:14.181489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:33719)
0811 15:11:14.186167 (+ 4678us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:14.186224 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4588}
I20260811 15:11:14.191736 23431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:14.191303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:48197)
0811 15:11:14.191582 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:14.191641 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":100,"threads_started":1}
I20260811 15:11:14.383471 23432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:14.383038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:48767)
0811 15:11:14.383316 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:14.383372 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":107,"threads_started":1}
W20260811 15:11:14.626843 22796 consensus_peers.cc:487] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 -> Peer 2ca3edc257ee4a9bb26aa80b674385ba (127.30.8.3:43089): Couldn't send request to peer 2ca3edc257ee4a9bb26aa80b674385ba. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e. This is attempt 1: this message will repeat every 5th retry.
I20260811 15:11:14.665974 23132 raft_consensus.cc:1261] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P e393586d25cb4b19a6ffdcc9e9f2a55c [term 2 FOLLOWER]: Refusing update from remote peer eda565b102f448078de510d0a0e548f0: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260811 15:11:14.666735 23425 consensus_queue.cc:1035] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 15:11:14.668747 23431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:14.668487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:54635)
0811 15:11:14.668620 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:14.668672 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 15:11:14.671056 23428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:14.670782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:51185)
0811 15:11:14.670901 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:14.670982 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 15:11:14.671684 23431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:14.671481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:57075)
0811 15:11:14.671582 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:14.671626 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 15:11:15.047959 23438 ts_tablet_manager.cc:927] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: Initiating tablet copy from peer eda565b102f448078de510d0a0e548f0 (127.30.8.1:40023)
I20260811 15:11:15.048579 23438 tablet_copy_client.cc:316] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: tablet copy: Beginning tablet copy session from remote peer at address 127.30.8.1:40023
Aug 11 15:11:15 dist-test-slave-1vmg krb5kdc[20707](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461073, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.1@KRBTEST.COM
I20260811 15:11:15.058553 23441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:15.049118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:40023 (local address 127.30.8.3:51223)
0811 15:11:15.049390 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:15.049410 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:15.049536 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:15.050061 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:15.050066 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:15.050464 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:15.050582 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:15.050588 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:15.051646 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:15.051649 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:15.052319 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:15.052328 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:15.052837 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:15.052839 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:15.052913 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:15.053430 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:15.053442 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:15.054848 (+ 1406us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:15.056651 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:15.056659 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:15.056664 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:15.056936 (+ 272us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:15.057271 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:15.057275 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:15.057277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:15.057377 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:15.057954 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:15.057961 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:15.058105 (+ 144us) client_negotiation.cc:769] Sending connection context
0811 15:11:15.058258 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 15:11:15.058410 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":110,"threads_started":1}
I20260811 15:11:15.058615 23446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:15.049296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:51223 (local address 127.30.8.1:40023)
0811 15:11:15.049665 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:15.049672 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:15.049704 (+ 32us) server_negotiation.cc:407] Connection header received
0811 15:11:15.049752 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:15.049756 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:15.049934 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:15.050052 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:15.050722 (+ 670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:15.051545 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:15.052411 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:15.052755 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:15.052831 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:15.055008 (+ 2177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:15.055019 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:15.055033 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:15.055058 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:15.056558 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:15.057050 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:15.057057 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:15.057059 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:15.057171 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:15.057471 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:15.057475 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:15.057477 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:15.057846 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:15.057992 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:15.058345 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 15:11:15.058483 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283,"thread_start_us":85,"threads_started":1}
I20260811 15:11:15.059711 22942 tablet_copy_service.cc:140] P eda565b102f448078de510d0a0e548f0: Received BeginTabletCopySession request for tablet cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e from peer 2ca3edc257ee4a9bb26aa80b674385ba ({username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:51223)
I20260811 15:11:15.059894 22942 tablet_copy_service.cc:161] P eda565b102f448078de510d0a0e548f0: Beginning new tablet copy session on tablet cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e from peer 2ca3edc257ee4a9bb26aa80b674385ba at {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:51223: session id = 2ca3edc257ee4a9bb26aa80b674385ba-cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e
I20260811 15:11:15.061120 22942 tablet_copy_source_session.cc:205] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P eda565b102f448078de510d0a0e548f0: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 15:11:15.065380 23438 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:15.072073 23438 tablet_copy_client.cc:791] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: tablet copy: Starting download of 0 data blocks...
I20260811 15:11:15.072276 23438 tablet_copy_client.cc:655] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: tablet copy: Starting download of 1 WAL segments...
I20260811 15:11:15.074064 23438 tablet_copy_client.cc:531] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 15:11:15.075903 23438 tablet_bootstrap.cc:492] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: Bootstrap starting.
I20260811 15:11:15.081317 23438 log.cc:824] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:15.088258 23438 tablet_bootstrap.cc:492] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 15:11:15.088572 23438 tablet_bootstrap.cc:492] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: Bootstrap complete.
I20260811 15:11:15.088907 23438 ts_tablet_manager.cc:1397] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: Time spent bootstrapping tablet: real 0.013s user 0.012s sys 0.000s
I20260811 15:11:15.091320 23438 raft_consensus.cc:348] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } } peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } }
I20260811 15:11:15.091571 23438 raft_consensus.cc:729] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ca3edc257ee4a9bb26aa80b674385ba, State: Initialized, Role: FOLLOWER
I20260811 15:11:15.092011 23438 consensus_queue.cc:260] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e393586d25cb4b19a6ffdcc9e9f2a55c" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 41153 } } peers { permanent_uuid: "2ca3edc257ee4a9bb26aa80b674385ba" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43089 } } peers { permanent_uuid: "eda565b102f448078de510d0a0e548f0" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40023 } }
I20260811 15:11:15.092896 23438 ts_tablet_manager.cc:1428] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba: Time spent starting tablet: real 0.004s user 0.005s sys 0.000s
I20260811 15:11:15.093183 23441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:15.092929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:54647)
0811 15:11:15.093048 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:15.093104 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 15:11:15.093642 22942 tablet_copy_service.cc:342] P eda565b102f448078de510d0a0e548f0: Request end of tablet copy session 2ca3edc257ee4a9bb26aa80b674385ba-cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e received from {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:51223
I20260811 15:11:15.093781 22942 tablet_copy_service.cc:433] P eda565b102f448078de510d0a0e548f0: ending tablet copy session 2ca3edc257ee4a9bb26aa80b674385ba-cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e on tablet cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e with peer 2ca3edc257ee4a9bb26aa80b674385ba
I20260811 15:11:15.094002 23441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:15.093597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:60395)
0811 15:11:15.093889 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:15.093941 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220}
I20260811 15:11:15.670584 23454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:15.670193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:48425)
0811 15:11:15.670408 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:15.670486 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":64,"threads_started":1}
I20260811 15:11:15.670791 23337 raft_consensus.cc:1203] T cacbfb8b2c2a4ee2bf9f2ee7ec5cf61e P 2ca3edc257ee4a9bb26aa80b674385ba [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260811 15:11:15.672170 23457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:15.671752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:41247)
0811 15:11:15.672027 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:15.672084 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":59,"threads_started":1}
I20260811 15:11:15.672652 23458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:15.672294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:51095)
0811 15:11:15.672508 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:15.672565 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":59,"threads_started":1}
I20260811 15:11:16.671942 23460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:16.671450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:56877)
0811 15:11:16.671771 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:16.671838 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":99,"threads_started":1}
I20260811 15:11:16.673405 23463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:16.673002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:54769)
0811 15:11:16.673244 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:16.673302 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":69,"threads_started":1}
I20260811 15:11:16.673720 23466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:16.673269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:47749)
0811 15:11:16.673579 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:16.673641 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":73,"threads_started":1}
I20260811 15:11:17.673166 23468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:17.672750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:52213)
0811 15:11:17.673009 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:17.673062 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":91,"threads_started":1}
I20260811 15:11:17.674568 23469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:17.674100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:36347)
0811 15:11:17.674401 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:17.674466 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":66,"threads_started":1}
I20260811 15:11:17.674743 23472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:17.674360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:41611)
0811 15:11:17.674602 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:17.674649 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":80,"threads_started":1}
I20260811 15:11:18.674599 23473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:18.674140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:37363)
0811 15:11:18.674436 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:18.674498 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":102,"threads_started":1}
I20260811 15:11:18.675325 23473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:18.675104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:54833)
0811 15:11:18.675213 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:18.675257 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260811 15:11:18.675897 23475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:18.675499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:47095)
0811 15:11:18.675747 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:18.675806 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":84,"threads_started":1}
I20260811 15:11:18.675920 23476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:18.675481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:54899)
0811 15:11:18.675796 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:18.675844 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":92,"threads_started":1}
I20260811 15:11:19.676906 23477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:19.676393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:49471)
0811 15:11:19.676714 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:19.676786 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":100,"threads_started":1}
I20260811 15:11:19.677063 23480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:19.676647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:60115)
0811 15:11:19.676924 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:19.676975 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":63,"threads_started":1}
I20260811 15:11:19.677167 23479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:19.676632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:41853)
0811 15:11:19.676886 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:19.676947 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":79,"threads_started":1}
I20260811 15:11:20.678298 23483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:20.677846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:38381)
0811 15:11:20.678124 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:20.678191 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":87,"threads_started":1}
I20260811 15:11:20.678377 23484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:20.677865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:55345)
0811 15:11:20.678212 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:20.678275 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":83,"threads_started":1}
I20260811 15:11:20.678377 23485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:20.677848 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:59653)
0811 15:11:20.678211 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:20.678275 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":102,"threads_started":1}
I20260811 15:11:21.679658 23486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:21.679216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:46677)
0811 15:11:21.679491 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:21.679555 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":85,"threads_started":1}
I20260811 15:11:21.679713 23488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:21.679219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:35219)
0811 15:11:21.679552 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:21.679612 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":95,"threads_started":1}
I20260811 15:11:21.679713 23489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:21.679226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:51321)
0811 15:11:21.679558 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:21.679612 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":82,"threads_started":1}
I20260811 15:11:22.681216 23491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:22.680656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:57191)
0811 15:11:22.681072 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:22.681126 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":132,"threads_started":1}
I20260811 15:11:22.681195 23494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:22.680695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:46465)
0811 15:11:22.680975 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:22.681063 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":197,"thread_start_us":93,"threads_started":1}
I20260811 15:11:22.681226 23495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:22.680798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:37739)
0811 15:11:22.681059 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:22.681125 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":63,"threads_started":1}
I20260811 15:11:23.682612 23497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:23.682175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:37179)
0811 15:11:23.682446 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:23.682506 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":88,"threads_started":1}
I20260811 15:11:23.682686 23499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:23.682185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:43447)
0811 15:11:23.682527 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:23.682579 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":87,"threads_started":1}
I20260811 15:11:23.682686 23500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:23.682185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:60389)
0811 15:11:23.682512 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:23.682579 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":84,"threads_started":1}
I20260811 15:11:24.684008 23501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:24.683537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:45405)
0811 15:11:24.683819 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:24.683889 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":94,"threads_started":1}
I20260811 15:11:24.684098 23502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:24.683536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:35971)
0811 15:11:24.683928 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:24.683990 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":95,"threads_started":1}
I20260811 15:11:24.684442 23504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:24.683971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:59111)
0811 15:11:24.684311 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:24.684365 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":80,"threads_started":1}
I20260811 15:11:25.685412 23507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:25.684840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:34641)
0811 15:11:25.685214 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:25.685281 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":105,"threads_started":1}
I20260811 15:11:25.685590 23511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:25.685141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:48403)
0811 15:11:25.685435 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:25.685483 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":76,"threads_started":1}
I20260811 15:11:25.685632 23509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:25.684871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:57297)
0811 15:11:25.685318 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:25.685423 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":84,"threads_started":1}
I20260811 15:11:26.686684 23513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:26.686265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:47031)
0811 15:11:26.686500 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:26.686573 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":68,"threads_started":1}
I20260811 15:11:26.686709 23514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:26.686293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:42625)
0811 15:11:26.686560 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:26.686610 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":64,"threads_started":1}
I20260811 15:11:26.686679 23512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:26.686167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:50679)
0811 15:11:26.686518 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:26.686572 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":86,"threads_started":1}
I20260811 15:11:27.688100 23515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:27.687624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.3:55461)
0811 15:11:27.687910 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:27.687975 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":89,"threads_started":1}
I20260811 15:11:27.688148 23519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:27.687626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.1:33521)
0811 15:11:27.687975 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:27.688038 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":90,"threads_started":1}
I20260811 15:11:27.688148 23518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:27.687626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45969 (local address 127.30.8.2:56777)
0811 15:11:27.687975 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:27.688038 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:45969: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":87,"threads_started":1}
I20260811 15:11:28.383263 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 22718
I20260811 15:11:28.404624 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 22998
I20260811 15:11:28.420142 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 23208
2026-08-11T15:11:28Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19394 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:11:28 dist-test-slave-1vmg krb5kdc[23522](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:11:28 dist-test-slave-1vmg krb5kdc[23522](info): set up 2 sockets
Aug 11 15:11:28 dist-test-slave-1vmg krb5kdc[23522](info): commencing operation
krb5kdc: starting...
W20260811 15:11:29.625627 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.003s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:11:29 dist-test-slave-1vmg krb5kdc[23522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:11:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:11:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:11:29.775609 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:32847
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:42089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:32847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:11:29.900532 23547 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:29.900879 23547 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:29.900942 23547 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:29.905241 23547 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 15:11:29.905320 23547 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 15:11:29.905342 23547 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 15:11:29.905359 23547 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:11:29.905383 23547 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:29.905411 23547 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:29.905437 23547 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:11:29.905468 23547 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:11:29.911326 23547 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:42089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:32847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:32847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23547
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:29.912628 23547 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:29.913769 23547 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 15:11:29.923456 23547 server_base.cc:1034] running on GCE node
W20260811 15:11:29.923426 23558 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:29.923524 23559 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:29.923426 23561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:29.926230 23547 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:29.927765 23547 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:29.929093 23547 hybrid_clock.cc:648] HybridClock initialized: now 1786461089929085 us; error 72 us; skew 500 ppm
Aug 11 15:11:29 dist-test-slave-1vmg krb5kdc[23522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461089, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:29.932377 23547 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:11:29.933490 23547 webserver.cc:460] Webserver started at http://127.30.8.62:34339/ using document root <none> and password file <none>
I20260811 15:11:29.934103 23547 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:29.934171 23547 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:29.934387 23547 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:11:29.936691 23547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "64fbe56d259a4dbbbcc8a4375878dcb1"
format_stamp: "Formatted at 2026-08-11 15:11:29 on dist-test-slave-1vmg"
server_key: "d672c12bbd6edc2fa7b1975f7845e1b3"
server_key_iv: "e07674c1c899f30359af879d41e5e1ff"
server_key_version: "encryptionkey@0"
I20260811 15:11:29.937315 23547 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "64fbe56d259a4dbbbcc8a4375878dcb1"
format_stamp: "Formatted at 2026-08-11 15:11:29 on dist-test-slave-1vmg"
server_key: "d672c12bbd6edc2fa7b1975f7845e1b3"
server_key_iv: "e07674c1c899f30359af879d41e5e1ff"
server_key_version: "encryptionkey@0"
I20260811 15:11:29.941514 23547 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:11:29.944262 23576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:29.945569 23547 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 15:11:29.945700 23547 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "64fbe56d259a4dbbbcc8a4375878dcb1"
format_stamp: "Formatted at 2026-08-11 15:11:29 on dist-test-slave-1vmg"
server_key: "d672c12bbd6edc2fa7b1975f7845e1b3"
server_key_iv: "e07674c1c899f30359af879d41e5e1ff"
server_key_version: "encryptionkey@0"
I20260811 15:11:29.945829 23547 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:29.967473 23547 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:29.970573 23547 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:29.970777 23547 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:29.978309 23547 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:32847
I20260811 15:11:29.978609 23655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:32847 every 8 connection(s)
I20260811 15:11:29.979465 23547 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:11:29.982681 23658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:29.986984 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 23547
I20260811 15:11:29.987146 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:11:29.987473 30752 external_mini_cluster.cc:1428] Setting key fc58eb019744f6058d9bbd75526fcb99
I20260811 15:11:29.989645 23658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: Bootstrap starting.
I20260811 15:11:29.993376 23658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: Neither blocks nor log segments found. Creating new log.
I20260811 15:11:29.994437 23658 log.cc:824] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:29.997213 23658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: No bootstrap required, opened a new log
Aug 11 15:11:29 dist-test-slave-1vmg krb5kdc[23522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461089, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:30.001472 23658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 32847 } }
I20260811 15:11:30.001819 23658 raft_consensus.cc:374] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:30.001891 23658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64fbe56d259a4dbbbcc8a4375878dcb1, State: Initialized, Role: FOLLOWER
I20260811 15:11:30.002507 23658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 32847 } }
I20260811 15:11:30.002686 23658 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:11:30.002763 23658 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:11:30.002876 23658 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:30.004175 23658 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 32847 } }
I20260811 15:11:30.004683 23658 leader_election.cc:302] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64fbe56d259a4dbbbcc8a4375878dcb1; no voters:
I20260811 15:11:30.005206 23658 leader_election.cc:288] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:11:30.005388 23666 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:11:30.005745 23666 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 1 LEADER]: Becoming Leader. State: Replica: 64fbe56d259a4dbbbcc8a4375878dcb1, State: Running, Role: LEADER
I20260811 15:11:30.006206 23666 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 32847 } }
I20260811 15:11:30.006490 23664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:29.989146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59002 (local address 127.30.8.62:32847)
0811 15:11:29.989647 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:29.989664 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:29.989704 (+ 40us) server_negotiation.cc:407] Connection header received
0811 15:11:29.990853 (+ 1149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:29.990895 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:29.991529 (+ 634us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:29.992017 (+ 488us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:29.993102 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:29.994362 (+ 1260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:29.995417 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:29.996055 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:29.996237 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:29.999146 (+ 2909us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:29.999154 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:29.999177 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:29.999216 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:30.002232 (+ 3016us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:30.002960 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:30.002969 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:30.002977 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:30.003133 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:30.003436 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:30.003440 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:30.003441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:30.004000 (+ 559us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:30.004245 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:30.004675 (+ 430us) server_negotiation.cc:299] Negotiation successful
0811 15:11:30.005148 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":155,"threads_started":1}
I20260811 15:11:30.006995 23658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:11:30.008293 23667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 32847 } } }
I20260811 15:11:30.008848 23667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [sys.catalog]: This master's current role is: LEADER
I20260811 15:11:30.009217 23667 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64fbe56d259a4dbbbcc8a4375878dcb1. Latest consensus state: current_term: 1 leader_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 32847 } } }
I20260811 15:11:30.009373 23667 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [sys.catalog]: This master's current role is: LEADER
W20260811 15:11:30.014096 23699 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:11:30.014277 23699 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:11:30.014427 23693 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:11:30.015473 23693 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:11:30.022274 23693 catalog_manager.cc:1347] Generated new cluster ID: 5859d57c426a43058f904a91b8f67730
I20260811 15:11:30.022375 23693 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:11:30.030658 23693 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:11:30.032131 23693 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:11:30.042284 23693 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: Generated new TSK 0
I20260811 15:11:30.043339 23693 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
I20260811 15:11:30.087378 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:32847
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:42089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:11:30.230844 23706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:30.231280 23706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:30.231350 23706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:30.235777 23706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:30.235877 23706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:30.235972 23706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:11:30.242151 23706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:42089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:32847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:30.243749 23706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:30.245144 23706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:30.260181 23723 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:30.260181 23720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:30.260281 23706 server_base.cc:1034] running on GCE node
W20260811 15:11:30.260181 23721 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:30.261006 23706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:30.261794 23706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:30.263036 23706 hybrid_clock.cc:648] HybridClock initialized: now 1786461090262999 us; error 48 us; skew 500 ppm
Aug 11 15:11:30 dist-test-slave-1vmg krb5kdc[23522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461090, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:30.266185 23706 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:11:30.267447 23706 webserver.cc:460] Webserver started at http://127.30.8.1:41795/ using document root <none> and password file <none>
I20260811 15:11:30.268204 23706 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:30.268280 23706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:30.268491 23706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:11:30.270812 23706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "ef373468f4e14efb844bde0ccc564a98"
format_stamp: "Formatted at 2026-08-11 15:11:30 on dist-test-slave-1vmg"
server_key: "da47c577936bebe67f552bd72e076e67"
server_key_iv: "84ffdc38a2eda382ecaefd86b27c3b22"
server_key_version: "encryptionkey@0"
I20260811 15:11:30.271472 23706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "ef373468f4e14efb844bde0ccc564a98"
format_stamp: "Formatted at 2026-08-11 15:11:30 on dist-test-slave-1vmg"
server_key: "da47c577936bebe67f552bd72e076e67"
server_key_iv: "84ffdc38a2eda382ecaefd86b27c3b22"
server_key_version: "encryptionkey@0"
I20260811 15:11:30.275853 23706 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:11:30.278643 23733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:30.280081 23706 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:11:30.280208 23706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "ef373468f4e14efb844bde0ccc564a98"
format_stamp: "Formatted at 2026-08-11 15:11:30 on dist-test-slave-1vmg"
server_key: "da47c577936bebe67f552bd72e076e67"
server_key_iv: "84ffdc38a2eda382ecaefd86b27c3b22"
server_key_version: "encryptionkey@0"
I20260811 15:11:30.280335 23706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:30.298966 23706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:30.302565 23706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:30.302773 23706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:30.303489 23706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:11:30.304464 23706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:11:30.304519 23706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:30.304585 23706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:11:30.304612 23706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:30.314796 23706 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:45981
I20260811 15:11:30.314815 23930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:45981 every 8 connection(s)
I20260811 15:11:30.315927 23706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:11:30.318572 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 23706
I20260811 15:11:30.318728 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:11:30.319090 30752 external_mini_cluster.cc:1428] Setting key f06def5db941c1cc557f01fd042d444d
Aug 11 15:11:30 dist-test-slave-1vmg krb5kdc[23522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461090, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.8.2@KRBTEST.COM; defaulting to no policy
I20260811 15:11:30.331388 23664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:30.317976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:60883 (local address 127.30.8.62:32847)
0811 15:11:30.318142 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:30.318147 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:30.319227 (+ 1080us) server_negotiation.cc:407] Connection header received
0811 15:11:30.320813 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:30.320816 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:30.320861 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:30.320909 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:30.322580 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.323198 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.324434 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.324767 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.324880 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:30.327671 (+ 2791us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:30.327675 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:30.327677 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:30.327704 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:30.329164 (+ 1460us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:30.329704 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:30.329708 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:30.329709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:30.329792 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:30.330211 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:30.330213 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:30.330214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:30.330414 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:30.330490 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:30.331112 (+ 622us) server_negotiation.cc:299] Negotiation successful
0811 15:11:30.331243 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 15:11:30.332474 23934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:30.318392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:60883)
0811 15:11:30.319033 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:30.319095 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:30.320566 (+ 1471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:30.321123 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:30.321134 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:30.321652 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:30.322394 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:30.322415 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.323344 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.323348 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:30.324304 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:30.324316 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.324968 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.324973 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:30.325165 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:30.325809 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:30.325827 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:30.327474 (+ 1647us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:30.329287 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:30.329296 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:30.329317 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:30.329602 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:30.329906 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:30.329910 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:30.329913 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:30.330121 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:30.330496 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:30.330503 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:30.330827 (+ 324us) client_negotiation.cc:769] Sending connection context
0811 15:11:30.331041 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 15:11:30.331364 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":121,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.2@KRBTEST.COM" created.
I20260811 15:11:30.333828 23931 heartbeater.cc:347] Connected to a master server at 127.30.8.62:32847
I20260811 15:11:30.334133 23931 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:30.334797 23931 heartbeater.cc:511] Master 127.30.8.62:32847 requested a full tablet report, sending...
I20260811 15:11:30.336817 23600 ts_manager.cc:194] Registered new tserver with Master: ef373468f4e14efb844bde0ccc564a98 (127.30.8.1:45981)
I20260811 15:11:30.338485 23600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:60883
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
I20260811 15:11:30.351241 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:32847
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:42089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:11:30.490104 23939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:30.490429 23939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:30.490475 23939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:30.494604 23939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:30.494693 23939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:30.494755 23939 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:11:30.500674 23939 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:42089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:32847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:30.501955 23939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:30.503154 23939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:30.514149 23950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:30.514266 23939 server_base.cc:1034] running on GCE node
W20260811 15:11:30.514173 23949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:30.514168 23952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:30.514941 23939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:30.515687 23939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:30.516884 23939 hybrid_clock.cc:648] HybridClock initialized: now 1786461090516861 us; error 35 us; skew 500 ppm
Aug 11 15:11:30 dist-test-slave-1vmg krb5kdc[23522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461090, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:30.520080 23939 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:11:30.521072 23939 webserver.cc:460] Webserver started at http://127.30.8.2:37559/ using document root <none> and password file <none>
I20260811 15:11:30.521692 23939 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:30.521765 23939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:30.521992 23939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:11:30.524292 23939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "49aa3b8cd9e34767976fc8af1559c06d"
format_stamp: "Formatted at 2026-08-11 15:11:30 on dist-test-slave-1vmg"
server_key: "5d39b1aa107227812c5832d7b7416251"
server_key_iv: "b90becb858c2f97f41033dde0ffa9c41"
server_key_version: "encryptionkey@0"
I20260811 15:11:30.524885 23939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "49aa3b8cd9e34767976fc8af1559c06d"
format_stamp: "Formatted at 2026-08-11 15:11:30 on dist-test-slave-1vmg"
server_key: "5d39b1aa107227812c5832d7b7416251"
server_key_iv: "b90becb858c2f97f41033dde0ffa9c41"
server_key_version: "encryptionkey@0"
I20260811 15:11:30.529412 23939 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:11:30.532176 23961 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:30.533535 23939 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 15:11:30.533664 23939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "49aa3b8cd9e34767976fc8af1559c06d"
format_stamp: "Formatted at 2026-08-11 15:11:30 on dist-test-slave-1vmg"
server_key: "5d39b1aa107227812c5832d7b7416251"
server_key_iv: "b90becb858c2f97f41033dde0ffa9c41"
server_key_version: "encryptionkey@0"
I20260811 15:11:30.533783 23939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:30.553635 23939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:30.556963 23939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:30.557175 23939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:30.557837 23939 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:11:30.558759 23939 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:11:30.558810 23939 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:30.558857 23939 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:11:30.558887 23939 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:30.567968 23939 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:40879
I20260811 15:11:30.567991 24157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:40879 every 8 connection(s)
I20260811 15:11:30.568969 23939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:11:30.570816 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 23939
I20260811 15:11:30.570972 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:11:30.571285 30752 external_mini_cluster.cc:1428] Setting key 77139b803a580dab067218fd9d6b487b
Aug 11 15:11:30 dist-test-slave-1vmg krb5kdc[23522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461090, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.8.3@KRBTEST.COM; defaulting to no policy
I20260811 15:11:30.585773 23664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:30.571042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:57365 (local address 127.30.8.62:32847)
0811 15:11:30.571181 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:30.571184 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:30.572300 (+ 1116us) server_negotiation.cc:407] Connection header received
0811 15:11:30.573906 (+ 1606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:30.573909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:30.573956 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:30.574022 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:30.575693 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.576465 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.578005 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.578405 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.578508 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:30.581648 (+ 3140us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:30.581653 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:30.581656 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:30.581688 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:30.583346 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:30.583978 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:30.583983 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:30.583984 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:30.584159 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:30.584532 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:30.584535 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:30.584536 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:30.584756 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:30.584843 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:30.585492 (+ 649us) server_negotiation.cc:299] Negotiation successful
0811 15:11:30.585635 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 15:11:30.586755 24162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:30.571577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:57365)
0811 15:11:30.572132 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:30.572190 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:30.573658 (+ 1468us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:30.574237 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:30.574247 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:30.574757 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:30.575530 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:30.575549 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.576586 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.576588 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:30.577849 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:30.577863 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.578658 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.578666 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:30.578903 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:30.579626 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:30.579646 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:30.581432 (+ 1786us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:30.583492 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:30.583498 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:30.583516 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:30.583812 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:30.584268 (+ 456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:30.584271 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:30.584273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:30.584433 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:30.584844 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:30.584850 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:30.585247 (+ 397us) client_negotiation.cc:769] Sending connection context
0811 15:11:30.585428 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 15:11:30.585753 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":128,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.3@KRBTEST.COM" created.
I20260811 15:11:30.588147 24159 heartbeater.cc:347] Connected to a master server at 127.30.8.62:32847
I20260811 15:11:30.588450 24159 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:30.589022 24159 heartbeater.cc:511] Master 127.30.8.62:32847 requested a full tablet report, sending...
I20260811 15:11:30.590303 23600 ts_manager.cc:194] Registered new tserver with Master: 49aa3b8cd9e34767976fc8af1559c06d (127.30.8.2:40879)
I20260811 15:11:30.590986 23600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:57365
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
I20260811 15:11:30.605777 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:32847
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:42089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:11:30.735859 24165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:30.736203 24165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:30.736268 24165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:30.740638 24165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:30.740731 24165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:30.740815 24165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:11:30.746522 24165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:42089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:32847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:30.747826 24165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:30.748914 24165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:30.759141 24176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:30.759157 24178 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:30.759141 24173 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:30.759230 24165 server_base.cc:1034] running on GCE node
I20260811 15:11:30.759867 24165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:30.760612 24165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:30.761798 24165 hybrid_clock.cc:648] HybridClock initialized: now 1786461090761780 us; error 36 us; skew 500 ppm
Aug 11 15:11:30 dist-test-slave-1vmg krb5kdc[23522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461090, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:30.764806 24165 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:11:30.765786 24165 webserver.cc:460] Webserver started at http://127.30.8.3:36103/ using document root <none> and password file <none>
I20260811 15:11:30.766369 24165 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:30.766430 24165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:30.766623 24165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:11:30.768831 24165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "e516bdb6543b44b1ba2cc2502b3316d4"
format_stamp: "Formatted at 2026-08-11 15:11:30 on dist-test-slave-1vmg"
server_key: "ad7cf47cdd29c85cd9f3eae1ea317edb"
server_key_iv: "76f282f4ca0ba0f64fdc3e3d63ca270d"
server_key_version: "encryptionkey@0"
I20260811 15:11:30.769397 24165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "e516bdb6543b44b1ba2cc2502b3316d4"
format_stamp: "Formatted at 2026-08-11 15:11:30 on dist-test-slave-1vmg"
server_key: "ad7cf47cdd29c85cd9f3eae1ea317edb"
server_key_iv: "76f282f4ca0ba0f64fdc3e3d63ca270d"
server_key_version: "encryptionkey@0"
I20260811 15:11:30.773936 24165 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 15:11:30.776703 24188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:30.778084 24165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:11:30.778224 24165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "e516bdb6543b44b1ba2cc2502b3316d4"
format_stamp: "Formatted at 2026-08-11 15:11:30 on dist-test-slave-1vmg"
server_key: "ad7cf47cdd29c85cd9f3eae1ea317edb"
server_key_iv: "76f282f4ca0ba0f64fdc3e3d63ca270d"
server_key_version: "encryptionkey@0"
I20260811 15:11:30.778347 24165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:30.801676 24165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:30.805176 24165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:30.805357 24165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:30.805956 24165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:11:30.806834 24165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:11:30.806896 24165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:30.806988 24165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:11:30.807013 24165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:30.816512 24165 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:36797
I20260811 15:11:30.816542 24373 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:36797 every 8 connection(s)
I20260811 15:11:30.817493 24165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:11:30.825083 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 24165
I20260811 15:11:30.825239 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:11:30.825523 30752 external_mini_cluster.cc:1428] Setting key 8756de56f703e276f3d9c0cbc01b54f1
Aug 11 15:11:30 dist-test-slave-1vmg krb5kdc[23522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461090, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:30.831871 23664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:30.819603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:57979 (local address 127.30.8.62:32847)
0811 15:11:30.819778 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:30.819783 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:30.820741 (+ 958us) server_negotiation.cc:407] Connection header received
0811 15:11:30.822081 (+ 1340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:30.822085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:30.822140 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:30.822219 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:30.823570 (+ 1351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.824123 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.825312 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.825562 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.825664 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:30.828599 (+ 2935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:30.828602 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:30.828605 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:30.828632 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:30.829965 (+ 1333us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:30.830462 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:30.830464 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:30.830465 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:30.830537 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:30.830804 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:30.830806 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:30.830807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:30.831044 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:30.831135 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:30.831629 (+ 494us) server_negotiation.cc:299] Negotiation successful
0811 15:11:30.831738 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 15:11:30.832818 24379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:30.820030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:57979)
0811 15:11:30.820568 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:30.820625 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:30.821842 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:30.822397 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:30.822403 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:30.822821 (+ 418us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:30.823435 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:30.823450 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.824232 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.824234 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:30.825194 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:30.825207 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.825741 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.825746 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:30.825946 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:30.826669 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:30.826692 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:30.828442 (+ 1750us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:30.830077 (+ 1635us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:30.830083 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:30.830097 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:30.830384 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:30.830599 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:30.830600 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:30.830602 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:30.830740 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:30.831131 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:30.831137 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:30.831425 (+ 288us) client_negotiation.cc:769] Sending connection context
0811 15:11:30.831598 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 15:11:30.831892 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":127,"threads_started":1}
I20260811 15:11:30.833920 24374 heartbeater.cc:347] Connected to a master server at 127.30.8.62:32847
I20260811 15:11:30.834185 24374 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:30.834739 24374 heartbeater.cc:511] Master 127.30.8.62:32847 requested a full tablet report, sending...
I20260811 15:11:30.835868 23600 ts_manager.cc:194] Registered new tserver with Master: e516bdb6543b44b1ba2cc2502b3316d4 (127.30.8.3:36797)
I20260811 15:11:30.836460 23600 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:57979
I20260811 15:11:30.839682 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 15:11:30.848417 23664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:30.841634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59014 (local address 127.30.8.62:32847)
0811 15:11:30.841783 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:30.841788 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:30.841927 (+ 139us) server_negotiation.cc:407] Connection header received
0811 15:11:30.842036 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:30.842039 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:30.842080 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:30.842150 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:30.842811 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.843377 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.844155 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.844357 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.844422 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:30.845705 (+ 1283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:30.845708 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:30.845711 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:30.845734 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:30.847047 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:30.847398 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:30.847400 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:30.847401 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:30.847474 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:30.847654 (+ 180us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:30.847656 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:30.847657 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:30.847880 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:30.847983 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:30.848153 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 15:11:30.848252 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 15:11:30.851570 23600 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59014:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:11:30.854861 23600 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:11:30.869625 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:30.864651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33986 (local address 127.30.8.3:36797)
0811 15:11:30.865040 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:30.865063 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:30.865090 (+ 27us) server_negotiation.cc:407] Connection header received
0811 15:11:30.865174 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:30.865179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:30.865400 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:30.865566 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:30.865881 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.866759 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.868281 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.868955 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.869065 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:30.869225 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:30.869364 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 15:11:30.869474 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":179,"threads_started":1}
I20260811 15:11:30.869614 24398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:30.863731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:36797 (local address 127.0.0.1:33986)
0811 15:11:30.864129 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:30.864181 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:30.864319 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:30.865562 (+ 1243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:30.865568 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:30.865592 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:30.865747 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:30.865755 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.866889 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.866893 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:30.868144 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:30.868155 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.869085 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.869088 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:30.869169 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:30.869202 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 15:11:30.869301 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 15:11:30.869440 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"mutex_wait_us":3,"thread_start_us":114,"threads_started":1}
I20260811 15:11:30.870229 24402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:30.864114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:45981 (local address 127.0.0.1:43678)
0811 15:11:30.865270 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:30.865282 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:30.865373 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:30.865966 (+ 593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:30.865969 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:30.865986 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:30.866060 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:30.866066 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.867411 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.867414 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:30.868613 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:30.868624 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.869803 (+ 1179us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.869807 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:30.869896 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:30.869923 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 15:11:30.870002 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 15:11:30.870093 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1098,"mutex_wait_us":42,"thread_start_us":123,"threads_started":1}
I20260811 15:11:30.870304 24403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:30.864596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43678 (local address 127.30.8.1:45981)
0811 15:11:30.864848 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:30.864856 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:30.865375 (+ 519us) server_negotiation.cc:407] Connection header received
0811 15:11:30.865466 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:30.865470 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:30.865677 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:30.865830 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:30.866190 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.867295 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.868745 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.869505 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.869610 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:30.869740 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:30.870048 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 15:11:30.870166 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":66,"threads_started":1}
I20260811 15:11:30.870806 24400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:30.863821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:40879 (local address 127.0.0.1:40070)
0811 15:11:30.865104 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:30.865118 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:30.865215 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:30.865772 (+ 557us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:30.865775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:30.865795 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:30.865902 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:30.865909 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.867290 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.867294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:30.868612 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:30.868624 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.870408 (+ 1784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.870412 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:30.870497 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:30.870521 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 15:11:30.870603 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 15:11:30.870679 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1216,"mutex_wait_us":33,"thread_start_us":103,"threads_started":1}
I20260811 15:11:30.871378 24405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:30.864943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40070 (local address 127.30.8.2:40879)
0811 15:11:30.865363 (+ 420us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:30.865374 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:30.865393 (+ 19us) server_negotiation.cc:407] Connection header received
0811 15:11:30.865465 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:30.865469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:30.865645 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:30.865789 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:30.865992 (+ 203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.867133 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.869305 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.870106 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.870201 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:30.870340 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:30.871136 (+ 796us) server_negotiation.cc:299] Negotiation successful
0811 15:11:30.871240 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":148,"threads_started":1}
I20260811 15:11:30.873435 23814 tablet_service.cc:1467] Processing CreateTablet for tablet 58862b8186da42af850d1d3ae2444783 (DEFAULT_TABLE table=test-table [id=3a69dc39a7674084b7f8a15aa5151423]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:11:30.873435 24262 tablet_service.cc:1467] Processing CreateTablet for tablet 58862b8186da42af850d1d3ae2444783 (DEFAULT_TABLE table=test-table [id=3a69dc39a7674084b7f8a15aa5151423]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:11:30.873713 24047 tablet_service.cc:1467] Processing CreateTablet for tablet 58862b8186da42af850d1d3ae2444783 (DEFAULT_TABLE table=test-table [id=3a69dc39a7674084b7f8a15aa5151423]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:11:30.874676 24262 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58862b8186da42af850d1d3ae2444783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:30.874670 23814 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58862b8186da42af850d1d3ae2444783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:30.874688 24047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58862b8186da42af850d1d3ae2444783. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:30.881460 24406 tablet_bootstrap.cc:492] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d: Bootstrap starting.
I20260811 15:11:30.882957 24409 tablet_bootstrap.cc:492] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4: Bootstrap starting.
I20260811 15:11:30.882957 24408 tablet_bootstrap.cc:492] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98: Bootstrap starting.
I20260811 15:11:30.884666 24406 tablet_bootstrap.cc:654] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d: Neither blocks nor log segments found. Creating new log.
I20260811 15:11:30.885083 24409 tablet_bootstrap.cc:654] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4: Neither blocks nor log segments found. Creating new log.
I20260811 15:11:30.885088 24408 tablet_bootstrap.cc:654] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98: Neither blocks nor log segments found. Creating new log.
I20260811 15:11:30.885641 24406 log.cc:824] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:30.885913 24408 log.cc:824] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:30.886404 24409 log.cc:824] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:30.888136 24409 tablet_bootstrap.cc:492] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4: No bootstrap required, opened a new log
I20260811 15:11:30.888140 24406 tablet_bootstrap.cc:492] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d: No bootstrap required, opened a new log
I20260811 15:11:30.888140 24408 tablet_bootstrap.cc:492] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98: No bootstrap required, opened a new log
I20260811 15:11:30.888343 24409 ts_tablet_manager.cc:1397] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 15:11:30.888343 24406 ts_tablet_manager.cc:1397] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 15:11:30.888343 24408 ts_tablet_manager.cc:1397] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 15:11:30.891665 24409 raft_consensus.cc:348] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:11:30.891939 24409 raft_consensus.cc:374] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:30.892017 24409 raft_consensus.cc:729] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e516bdb6543b44b1ba2cc2502b3316d4, State: Initialized, Role: FOLLOWER
I20260811 15:11:30.892560 24409 consensus_queue.cc:260] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:11:30.893321 24406 raft_consensus.cc:348] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:11:30.893581 24408 raft_consensus.cc:348] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:11:30.893885 24408 raft_consensus.cc:374] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:30.893889 24406 raft_consensus.cc:374] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:30.893968 24406 raft_consensus.cc:729] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49aa3b8cd9e34767976fc8af1559c06d, State: Initialized, Role: FOLLOWER
I20260811 15:11:30.893997 24408 raft_consensus.cc:729] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef373468f4e14efb844bde0ccc564a98, State: Initialized, Role: FOLLOWER
I20260811 15:11:30.894299 24409 ts_tablet_manager.cc:1428] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 15:11:30.894678 24408 consensus_queue.cc:260] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:11:30.894678 24406 consensus_queue.cc:260] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:11:30.894830 24374 heartbeater.cc:503] Master 127.30.8.62:32847 was elected leader, sending a full tablet report...
I20260811 15:11:30.895573 24159 heartbeater.cc:503] Master 127.30.8.62:32847 was elected leader, sending a full tablet report...
I20260811 15:11:30.895828 23931 heartbeater.cc:503] Master 127.30.8.62:32847 was elected leader, sending a full tablet report...
I20260811 15:11:30.896714 24406 ts_tablet_manager.cc:1428] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260811 15:11:30.897586 24408 ts_tablet_manager.cc:1428] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260811 15:11:30.951049 24420 raft_consensus.cc:482] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:11:30.951431 24420 raft_consensus.cc:504] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:11:30.952997 24420 leader_election.cc:288] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e516bdb6543b44b1ba2cc2502b3316d4 (127.30.8.3:36797), 49aa3b8cd9e34767976fc8af1559c06d (127.30.8.2:40879)
I20260811 15:11:30.958810 24404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:30.953461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:39167 (local address 127.30.8.3:36797)
0811 15:11:30.953614 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:30.953620 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:30.954035 (+ 415us) server_negotiation.cc:407] Connection header received
0811 15:11:30.954228 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:30.954231 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:30.954285 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:30.954358 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:30.954739 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.955496 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.957237 (+ 1741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.957919 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.958196 (+ 277us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:30.958278 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:30.958574 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 15:11:30.958655 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 15:11:30.958810 24405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:30.953561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:58089 (local address 127.30.8.2:40879)
0811 15:11:30.953720 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:30.953725 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:30.954171 (+ 446us) server_negotiation.cc:407] Connection header received
0811 15:11:30.954330 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:30.954333 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:30.954385 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:30.954460 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:30.954775 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.955367 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.957273 (+ 1906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.957934 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.958196 (+ 262us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:30.958278 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:30.958568 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 15:11:30.958655 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 15:11:30.958897 24423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:30.953424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:40879 (local address 127.30.8.1:58089)
0811 15:11:30.954049 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:30.954062 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:30.954207 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:30.954480 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:30.954483 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:30.954499 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:30.954661 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:30.954667 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.955528 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.955531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:30.957078 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:30.957091 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.958196 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.958201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:30.958412 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:30.958455 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 15:11:30.958546 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 15:11:30.958688 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":156,"thread_start_us":224,"threads_started":1}
I20260811 15:11:30.958899 24422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:30.953419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:36797 (local address 127.30.8.1:39167)
0811 15:11:30.953910 (+ 491us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:30.953927 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:30.954104 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:30.954389 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:30.954392 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:30.954414 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:30.954616 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:30.954623 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.955631 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.955634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:30.957078 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:30.957091 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.958196 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.958201 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:30.958417 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:30.958461 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 15:11:30.958557 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 15:11:30.958688 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":229,"thread_start_us":236,"threads_started":1}
I20260811 15:11:30.959815 24075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58862b8186da42af850d1d3ae2444783" candidate_uuid: "ef373468f4e14efb844bde0ccc564a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49aa3b8cd9e34767976fc8af1559c06d" is_pre_election: true
I20260811 15:11:30.959919 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58862b8186da42af850d1d3ae2444783" candidate_uuid: "ef373468f4e14efb844bde0ccc564a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" is_pre_election: true
I20260811 15:11:30.960191 24075 raft_consensus.cc:2454] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef373468f4e14efb844bde0ccc564a98 in term 0.
I20260811 15:11:30.960206 24300 raft_consensus.cc:2454] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef373468f4e14efb844bde0ccc564a98 in term 0.
I20260811 15:11:30.960724 23736 leader_election.cc:302] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49aa3b8cd9e34767976fc8af1559c06d, ef373468f4e14efb844bde0ccc564a98; no voters:
I20260811 15:11:30.961037 24420 raft_consensus.cc:2781] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:11:30.961134 24420 raft_consensus.cc:482] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:11:30.961185 24420 raft_consensus.cc:3037] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:30.962435 24420 raft_consensus.cc:504] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:11:30.962841 24420 leader_election.cc:288] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [CANDIDATE]: Term 1 election: Requested vote from peers e516bdb6543b44b1ba2cc2502b3316d4 (127.30.8.3:36797), 49aa3b8cd9e34767976fc8af1559c06d (127.30.8.2:40879)
I20260811 15:11:30.963428 24075 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58862b8186da42af850d1d3ae2444783" candidate_uuid: "ef373468f4e14efb844bde0ccc564a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49aa3b8cd9e34767976fc8af1559c06d"
I20260811 15:11:30.963593 24300 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58862b8186da42af850d1d3ae2444783" candidate_uuid: "ef373468f4e14efb844bde0ccc564a98" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e516bdb6543b44b1ba2cc2502b3316d4"
I20260811 15:11:30.963729 24300 raft_consensus.cc:3037] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:30.963665 24075 raft_consensus.cc:3037] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:30.964870 24300 raft_consensus.cc:2454] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef373468f4e14efb844bde0ccc564a98 in term 1.
I20260811 15:11:30.965256 24075 raft_consensus.cc:2454] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef373468f4e14efb844bde0ccc564a98 in term 1.
I20260811 15:11:30.965278 23737 leader_election.cc:302] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e516bdb6543b44b1ba2cc2502b3316d4, ef373468f4e14efb844bde0ccc564a98; no voters:
I20260811 15:11:30.965459 24420 raft_consensus.cc:2781] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:11:30.965734 24420 raft_consensus.cc:686] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [term 1 LEADER]: Becoming Leader. State: Replica: ef373468f4e14efb844bde0ccc564a98, State: Running, Role: LEADER
I20260811 15:11:30.966166 24420 consensus_queue.cc:237] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:11:30.969682 23600 catalog_manager.cc:5495] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 reported cstate change: term changed from 0 to 1, leader changed from <none> to ef373468f4e14efb844bde0ccc564a98 (127.30.8.1). New cstate: current_term: 1 leader_uuid: "ef373468f4e14efb844bde0ccc564a98" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } health_report { overall_health: HEALTHY } } }
I20260811 15:11:30.996681 24403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:30.992722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43682 (local address 127.30.8.1:45981)
0811 15:11:30.992896 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:30.992900 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:30.992917 (+ 17us) server_negotiation.cc:407] Connection header received
0811 15:11:30.993086 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:30.993089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:30.993170 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:30.993281 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:11:30.993575 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.994093 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.995048 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:30.995278 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:30.995359 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:30.995769 (+ 410us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:11:30.996272 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:11:30.996371 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:30.996465 (+ 94us) server_negotiation.cc:299] Negotiation successful
0811 15:11:30.996527 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 15:11:31.003135 24300 raft_consensus.cc:1261] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4 [term 1 FOLLOWER]: Refusing update from remote peer ef373468f4e14efb844bde0ccc564a98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:11:31.003135 24075 raft_consensus.cc:1261] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d [term 1 FOLLOWER]: Refusing update from remote peer ef373468f4e14efb844bde0ccc564a98: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:11:31.003855 24425 consensus_queue.cc:1035] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:11:31.004055 24420 consensus_queue.cc:1035] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:11:31.012672 24432 mvcc.cc:205] Tried to move back new op lower bound from 7317344628740726784 to 7317344628600635392. Current Snapshot: MvccSnapshot[applied={T|T < 7317344628740726784}]
I20260811 15:11:31.012928 24430 mvcc.cc:205] Tried to move back new op lower bound from 7317344628740726784 to 7317344628600635392. Current Snapshot: MvccSnapshot[applied={T|T < 7317344628740726784}]
I20260811 15:11:31.013716 24433 mvcc.cc:205] Tried to move back new op lower bound from 7317344628740726784 to 7317344628600635392. Current Snapshot: MvccSnapshot[applied={T|T < 7317344628740726784}]
I20260811 15:11:31.024498 23600 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:11:31.024709 23600 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59014:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:11:31.027477 23600 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: Sending DeleteTablet for 3 replicas of tablet 58862b8186da42af850d1d3ae2444783
I20260811 15:11:31.028434 24047 tablet_service.cc:1514] Processing DeleteTablet for tablet 58862b8186da42af850d1d3ae2444783 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:11:31 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:40070
I20260811 15:11:31.028434 24262 tablet_service.cc:1514] Processing DeleteTablet for tablet 58862b8186da42af850d1d3ae2444783 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:11:31 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:33986
I20260811 15:11:31.028790 23814 tablet_service.cc:1514] Processing DeleteTablet for tablet 58862b8186da42af850d1d3ae2444783 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:11:31 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:43678
I20260811 15:11:31.029072 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 23706
I20260811 15:11:31.029340 24444 tablet_replica.cc:330] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4: stopping tablet replica
I20260811 15:11:31.041591 24445 tablet_replica.cc:330] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d: stopping tablet replica
I20260811 15:11:31.042232 24444 raft_consensus.cc:2229] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:11:31.042554 24445 raft_consensus.cc:2229] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:11:31.043037 24445 raft_consensus.cc:2258] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:11:31.043365 24444 raft_consensus.cc:2258] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:11:31.045323 24444 ts_tablet_manager.cc:1905] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:11:31.045328 24445 ts_tablet_manager.cc:1905] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:11:31.047794 24444 ts_tablet_manager.cc:1918] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:11:31.047905 24444 log.cc:1196] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/58862b8186da42af850d1d3ae2444783
I20260811 15:11:31.048229 24444 ts_tablet_manager.cc:1939] T 58862b8186da42af850d1d3ae2444783 P e516bdb6543b44b1ba2cc2502b3316d4: Deleting consensus metadata
I20260811 15:11:31.049218 24445 ts_tablet_manager.cc:1918] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:11:31.049408 24445 log.cc:1196] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/58862b8186da42af850d1d3ae2444783
I20260811 15:11:31.049726 23584 catalog_manager.cc:4841] TS e516bdb6543b44b1ba2cc2502b3316d4 (127.30.8.3:36797): tablet 58862b8186da42af850d1d3ae2444783 (table test-table [id=3a69dc39a7674084b7f8a15aa5151423]) successfully deleted
I20260811 15:11:31.049914 24445 ts_tablet_manager.cc:1939] T 58862b8186da42af850d1d3ae2444783 P 49aa3b8cd9e34767976fc8af1559c06d: Deleting consensus metadata
I20260811 15:11:31.051690 23581 catalog_manager.cc:4841] TS 49aa3b8cd9e34767976fc8af1559c06d (127.30.8.2:40879): tablet 58862b8186da42af850d1d3ae2444783 (table test-table [id=3a69dc39a7674084b7f8a15aa5151423]) successfully deleted
W20260811 15:11:31.054245 23577 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.8.1:45981 (error 108)
I20260811 15:11:31.054347 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 23939
I20260811 15:11:31.066859 24400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:31.066447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:45981 (local address 127.0.0.1:43698)
0811 15:11:31.066642 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:31.066742 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.1:45981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
W20260811 15:11:31.067209 23577 catalog_manager.cc:4584] TS ef373468f4e14efb844bde0ccc564a98 (127.30.8.1:45981): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 58862b8186da42af850d1d3ae2444783: Network error: Client connection negotiation failed: client connection to 127.30.8.1:45981: connect: Connection refused (error 111)
I20260811 15:11:31.076247 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 24165
I20260811 15:11:31.099211 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 23547
I20260811 15:11:51.116508 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:32847
--webserver_interface=127.30.8.62
--webserver_port=34339
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:42089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:32847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:11:51.245186 24449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:51.245499 24449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:51.245543 24449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:51.249902 24449 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 15:11:51.249985 24449 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 15:11:51.250001 24449 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 15:11:51.250015 24449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:11:51.250033 24449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:51.250051 24449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:51.250082 24449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:11:51.250098 24449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:11:51.256036 24449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:42089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:32847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:32847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=34339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:51.257265 24449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:51.258309 24449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:51.273108 24459 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:51.273145 24460 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:51.273161 24469 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:51.273277 24449 server_base.cc:1034] running on GCE node
I20260811 15:11:51.274333 24449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:51.276027 24449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:51.277243 24449 hybrid_clock.cc:648] HybridClock initialized: now 1786461111277220 us; error 42 us; skew 500 ppm
Aug 11 15:11:51 dist-test-slave-1vmg krb5kdc[23522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461111, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:51.280540 24449 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:11:51.281661 24449 webserver.cc:460] Webserver started at http://127.30.8.62:34339/ using document root <none> and password file <none>
I20260811 15:11:51.282281 24449 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:51.282366 24449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:51.286736 24449 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 15:11:51.289162 24478 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:51.290521 24449 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:11:51.290673 24449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "64fbe56d259a4dbbbcc8a4375878dcb1"
format_stamp: "Formatted at 2026-08-11 15:11:29 on dist-test-slave-1vmg"
server_key: "d672c12bbd6edc2fa7b1975f7845e1b3"
server_key_iv: "e07674c1c899f30359af879d41e5e1ff"
server_key_version: "encryptionkey@0"
I20260811 15:11:51.291389 24449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:51.351706 24449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:51.355233 24449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:51.355443 24449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:51.363063 24449 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:32847
I20260811 15:11:51.363389 24568 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:32847 every 8 connection(s)
I20260811 15:11:51.364403 24449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:11:51.366351 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 24449
I20260811 15:11:51.366883 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:45981
--local_ip_for_outbound_sockets=127.30.8.1
--tserver_master_addrs=127.30.8.62:32847
--webserver_port=41795
--webserver_interface=127.30.8.1
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:42089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:11:51.370088 24571 sys_catalog.cc:263] Verifying existing consensus state
I20260811 15:11:51.372134 24571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: Bootstrap starting.
I20260811 15:11:51.389743 24571 log.cc:824] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:51.399250 24571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 15:11:51.399667 24571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: Bootstrap complete.
I20260811 15:11:51.403971 24571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 32847 } }
I20260811 15:11:51.404423 24571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64fbe56d259a4dbbbcc8a4375878dcb1, State: Initialized, Role: FOLLOWER
I20260811 15:11:51.404953 24571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 32847 } }
I20260811 15:11:51.405078 24571 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:11:51.405153 24571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:11:51.405257 24571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 1 FOLLOWER]: Advancing to term 2
I20260811 15:11:51.407413 24571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 32847 } }
I20260811 15:11:51.407774 24571 leader_election.cc:302] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64fbe56d259a4dbbbcc8a4375878dcb1; no voters:
I20260811 15:11:51.408102 24571 leader_election.cc:288] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 15:11:51.408243 24578 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 15:11:51.408455 24578 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 2 LEADER]: Becoming Leader. State: Replica: 64fbe56d259a4dbbbcc8a4375878dcb1, State: Running, Role: LEADER
I20260811 15:11:51.408862 24578 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 32847 } }
I20260811 15:11:51.409511 24571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:11:51.410429 24580 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64fbe56d259a4dbbbcc8a4375878dcb1. Latest consensus state: current_term: 2 leader_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 32847 } } }
I20260811 15:11:51.410561 24580 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [sys.catalog]: This master's current role is: LEADER
I20260811 15:11:51.410884 24579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 32847 } } }
I20260811 15:11:51.411010 24579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [sys.catalog]: This master's current role is: LEADER
I20260811 15:11:51.414698 24598 catalog_manager.cc:1259] Loaded cluster ID: 5859d57c426a43058f904a91b8f67730
I20260811 15:11:51.414762 24598 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: loading cluster ID for follower catalog manager: success
I20260811 15:11:51.416776 24598 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: acquiring CA information for follower catalog manager: success
I20260811 15:11:51.418089 24598 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 15:11:51.418215 24589 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:11:51.420360 24589 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:11:51.420785 24589 catalog_manager.cc:1259] Loaded cluster ID: 5859d57c426a43058f904a91b8f67730
I20260811 15:11:51.420835 24589 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:11:51.421417 24589 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:11:51.421835 24589 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: Loaded TSK: 0
I20260811 15:11:51.430491 24589 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: Generated new TSK 1
I20260811 15:11:51.430794 24589 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 15:11:51.513307 24573 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:51.513629 24573 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:51.513676 24573 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:51.518052 24573 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:51.518152 24573 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:51.518215 24573 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:11:51.524066 24573 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:42089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:45981
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=41795
--tserver_master_addrs=127.30.8.62:32847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24573
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:51.525308 24573 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:51.526401 24573 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:51.537240 24613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:51.537439 24616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:51.537634 24573 server_base.cc:1034] running on GCE node
W20260811 15:11:51.537266 24612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:51.538326 24573 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:51.539114 24573 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:51.540310 24573 hybrid_clock.cc:648] HybridClock initialized: now 1786461111540291 us; error 38 us; skew 500 ppm
Aug 11 15:11:51 dist-test-slave-1vmg krb5kdc[23522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461111, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:51.543709 24573 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:11:51.544768 24573 webserver.cc:460] Webserver started at http://127.30.8.1:41795/ using document root <none> and password file <none>
I20260811 15:11:51.545373 24573 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:51.545466 24573 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:51.549988 24573 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:11:51.552578 24632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:51.554064 24573 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 15:11:51.554194 24573 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "ef373468f4e14efb844bde0ccc564a98"
format_stamp: "Formatted at 2026-08-11 15:11:30 on dist-test-slave-1vmg"
server_key: "da47c577936bebe67f552bd72e076e67"
server_key_iv: "84ffdc38a2eda382ecaefd86b27c3b22"
server_key_version: "encryptionkey@0"
I20260811 15:11:51.554778 24573 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:51.583241 24573 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:51.586391 24573 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:51.586592 24573 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:51.587281 24573 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:11:51.588462 24647 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 15:11:51.593046 24573 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 15:11:51.593137 24573 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 15:11:51.593210 24573 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 15:11:51.595165 24573 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 15:11:51.595227 24573 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260811 15:11:51.595814 24647 tablet_bootstrap.cc:492] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98: Bootstrap starting.
I20260811 15:11:51.606040 24573 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:45981
I20260811 15:11:51.606221 24835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:45981 every 8 connection(s)
I20260811 15:11:51.607283 24573 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:11:51.616221 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 24573
I20260811 15:11:51.616675 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:40879
--local_ip_for_outbound_sockets=127.30.8.2
--tserver_master_addrs=127.30.8.62:32847
--webserver_port=37559
--webserver_interface=127.30.8.2
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:42089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:11:51.620853 24647 log.cc:824] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98: Log is configured to *not* fsync() on all Append() calls
Aug 11 15:11:51 dist-test-slave-1vmg krb5kdc[23522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461111, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:51.628611 24841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:51.609501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:39459)
0811 15:11:51.610054 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:51.610113 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:51.611076 (+ 963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:51.613022 (+ 1946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:51.613031 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:51.613448 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:51.614015 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:51.614030 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:51.616493 (+ 2463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:51.616498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:51.618150 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:51.618169 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:51.619401 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:51.619407 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:51.619649 (+ 242us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:51.620305 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:51.620325 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:51.621900 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:51.624866 (+ 2966us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:51.624872 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:51.624888 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:51.625258 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:51.625721 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:51.625724 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:51.625726 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:51.625900 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:51.626693 (+ 793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:51.626700 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:51.627026 (+ 326us) client_negotiation.cc:769] Sending connection context
0811 15:11:51.627229 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 15:11:51.627589 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":146,"threads_started":1}
I20260811 15:11:51.631146 24842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:51.609659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:39459 (local address 127.30.8.62:32847)
0811 15:11:51.610414 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:51.610441 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:51.610508 (+ 67us) server_negotiation.cc:407] Connection header received
0811 15:11:51.611695 (+ 1187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:51.611718 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:51.612331 (+ 613us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:51.612994 (+ 663us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:51.614268 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:51.616188 (+ 1920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:51.618423 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:51.619217 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:51.619427 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:51.622179 (+ 2752us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:51.622189 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:51.622209 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:51.622249 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:51.624703 (+ 2454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:51.625422 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:51.625427 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:51.625433 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:51.625601 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:51.626022 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:51.626025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:51.626027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:51.626557 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:51.629665 (+ 3108us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:51.629848 (+ 183us) server_negotiation.cc:299] Negotiation successful
0811 15:11:51.630214 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":211,"threads_started":1}
I20260811 15:11:51.631727 24647 tablet_bootstrap.cc:492] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 15:11:51.632146 24647 tablet_bootstrap.cc:492] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98: Bootstrap complete.
I20260811 15:11:51.633162 24647 ts_tablet_manager.cc:1397] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98: Time spent bootstrapping tablet: real 0.038s user 0.027s sys 0.008s
I20260811 15:11:51.634791 24836 heartbeater.cc:347] Connected to a master server at 127.30.8.62:32847
I20260811 15:11:51.635092 24836 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:51.635735 24836 heartbeater.cc:511] Master 127.30.8.62:32847 requested a full tablet report, sending...
I20260811 15:11:51.636255 24647 raft_consensus.cc:348] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:11:51.636806 24647 raft_consensus.cc:729] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef373468f4e14efb844bde0ccc564a98, State: Initialized, Role: FOLLOWER
I20260811 15:11:51.637342 24647 consensus_queue.cc:260] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:11:51.638677 24647 ts_tablet_manager.cc:1428] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98: Time spent starting tablet: real 0.005s user 0.006s sys 0.001s
I20260811 15:11:51.639248 24505 ts_manager.cc:194] Registered new tserver with Master: ef373468f4e14efb844bde0ccc564a98 (127.30.8.1:45981)
I20260811 15:11:51.643172 24505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:39459
I20260811 15:11:51.646129 24836 heartbeater.cc:503] Master 127.30.8.62:32847 was elected leader, sending a full tablet report...
I20260811 15:11:51.650069 24853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:51.642563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:45981 (local address 127.0.0.1:58954)
0811 15:11:51.643624 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:51.643697 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:51.643931 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:51.645403 (+ 1472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:51.645412 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:51.645451 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:51.645604 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:51.645613 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:51.647098 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:51.647101 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:51.648418 (+ 1317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:51.648430 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:51.649564 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:51.649566 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:51.649663 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:51.649713 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 15:11:51.649818 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 15:11:51.649938 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":706,"thread_start_us":169,"threads_started":1}
I20260811 15:11:51.650190 24854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:51.642751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58954 (local address 127.30.8.1:45981)
0811 15:11:51.644438 (+ 1687us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:51.644451 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:51.644481 (+ 30us) server_negotiation.cc:407] Connection header received
0811 15:11:51.644582 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:51.644589 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:51.645147 (+ 558us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:51.645338 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:51.645791 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:51.646952 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:51.648569 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:51.649431 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:51.649563 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:51.649743 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:51.649885 (+ 142us) server_negotiation.cc:299] Negotiation successful
0811 15:11:51.650005 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1563,"thread_start_us":211,"threads_started":1}
I20260811 15:11:51.651480 24716 tablet_service.cc:1514] Processing DeleteTablet for tablet 58862b8186da42af850d1d3ae2444783 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:11:31 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:58954
I20260811 15:11:51.652534 24856 tablet_replica.cc:330] stopping tablet replica
I20260811 15:11:51.652956 24856 raft_consensus.cc:2229] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:11:51.653296 24856 raft_consensus.cc:2258] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:11:51.654731 24856 ts_tablet_manager.cc:1905] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:11:51.660838 24856 ts_tablet_manager.cc:1918] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:11:51.661008 24856 log.cc:1196] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/wal/wals/58862b8186da42af850d1d3ae2444783
I20260811 15:11:51.661309 24856 ts_tablet_manager.cc:1939] T 58862b8186da42af850d1d3ae2444783 P ef373468f4e14efb844bde0ccc564a98: Deleting consensus metadata
I20260811 15:11:51.664072 24480 catalog_manager.cc:4841] TS ef373468f4e14efb844bde0ccc564a98 (127.30.8.1:45981): tablet 58862b8186da42af850d1d3ae2444783 (table test-table [id=3a69dc39a7674084b7f8a15aa5151423]) successfully deleted
W20260811 15:11:51.765401 24844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:51.765781 24844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:51.765839 24844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:51.770303 24844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:51.770399 24844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:51.770483 24844 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:11:51.777804 24844 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:42089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:40879
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=37559
--tserver_master_addrs=127.30.8.62:32847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:51.779232 24844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:51.780427 24844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:11:51.791178 24864 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:51.791181 24865 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:51.791244 24844 server_base.cc:1034] running on GCE node
W20260811 15:11:51.791178 24869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:51.791920 24844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:51.792649 24844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:51.793828 24844 hybrid_clock.cc:648] HybridClock initialized: now 1786461111793796 us; error 47 us; skew 500 ppm
Aug 11 15:11:51 dist-test-slave-1vmg krb5kdc[23522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461111, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:51.796802 24844 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:11:51.797868 24844 webserver.cc:460] Webserver started at http://127.30.8.2:37559/ using document root <none> and password file <none>
I20260811 15:11:51.798504 24844 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:51.798599 24844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:51.803026 24844 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 15:11:51.805408 24879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:51.806748 24844 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:11:51.806875 24844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "49aa3b8cd9e34767976fc8af1559c06d"
format_stamp: "Formatted at 2026-08-11 15:11:30 on dist-test-slave-1vmg"
server_key: "5d39b1aa107227812c5832d7b7416251"
server_key_iv: "b90becb858c2f97f41033dde0ffa9c41"
server_key_version: "encryptionkey@0"
I20260811 15:11:51.807546 24844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:51.818238 24844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:51.821796 24844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:51.822001 24844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:51.822679 24844 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:11:51.823644 24844 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:11:51.823709 24844 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:51.823768 24844 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:11:51.823799 24844 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:51.833055 24844 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:40879
I20260811 15:11:51.833091 25066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:40879 every 8 connection(s)
I20260811 15:11:51.834197 24844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:11:51.837803 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 24844
I20260811 15:11:51.838263 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:36797
--local_ip_for_outbound_sockets=127.30.8.3
--tserver_master_addrs=127.30.8.62:32847
--webserver_port=36103
--webserver_interface=127.30.8.3
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:42089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 15:11:51 dist-test-slave-1vmg krb5kdc[23522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461111, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:51.849887 24842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:51.836276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:52241 (local address 127.30.8.62:32847)
0811 15:11:51.836433 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:51.836437 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:51.837354 (+ 917us) server_negotiation.cc:407] Connection header received
0811 15:11:51.838808 (+ 1454us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:51.838811 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:51.838880 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:51.838999 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:51.840866 (+ 1867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:51.841509 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:51.842781 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:51.843071 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:51.843164 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:51.846023 (+ 2859us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:51.846026 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:51.846028 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:51.846058 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:51.847466 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:51.848086 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:51.848091 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:51.848093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:51.848203 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:51.848559 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:51.848562 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:51.848564 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:51.848858 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:51.848957 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:51.849535 (+ 578us) server_negotiation.cc:299] Negotiation successful
0811 15:11:51.849700 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 15:11:51.850766 25076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:51.836679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:52241)
0811 15:11:51.837183 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:51.837233 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:51.838560 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:51.839190 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:51.839212 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:51.839598 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:51.840691 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:51.840715 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:51.841635 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:51.841638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:51.842650 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:51.842661 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:51.843242 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:51.843247 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:51.843420 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:51.844067 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:51.844084 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:51.845829 (+ 1745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:51.847586 (+ 1757us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:51.847593 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:51.847614 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:51.847946 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:51.848308 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:51.848310 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:51.848311 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:51.848466 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:51.848968 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:51.848974 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:51.849277 (+ 303us) client_negotiation.cc:769] Sending connection context
0811 15:11:51.849457 (+ 180us) client_negotiation.cc:240] Negotiation successful
0811 15:11:51.849762 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":121,"threads_started":1}
I20260811 15:11:51.852008 25067 heartbeater.cc:347] Connected to a master server at 127.30.8.62:32847
I20260811 15:11:51.852275 25067 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:51.852842 25067 heartbeater.cc:511] Master 127.30.8.62:32847 requested a full tablet report, sending...
I20260811 15:11:51.854000 24505 ts_manager.cc:194] Registered new tserver with Master: 49aa3b8cd9e34767976fc8af1559c06d (127.30.8.2:40879)
I20260811 15:11:51.854786 24505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:52241
W20260811 15:11:51.978185 25080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:11:51.978567 25080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:11:51.978619 25080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:11:51.983028 25080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:11:51.983121 25080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:11:51.983191 25080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:11:51.989511 25080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:42089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:36797
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=36103
--tserver_master_addrs=127.30.8.62:32847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:11:51.990993 25080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:11:51.992295 25080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 15:11:52.004492 25080 server_base.cc:1034] running on GCE node
W20260811 15:11:52.004438 25085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:52.004438 25090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:11:52.004441 25086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:11:52.005228 25080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:11:52.006003 25080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:11:52.007170 25080 hybrid_clock.cc:648] HybridClock initialized: now 1786461112007139 us; error 47 us; skew 500 ppm
Aug 11 15:11:52 dist-test-slave-1vmg krb5kdc[23522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461112, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:11:52.015978 25080 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:11:52.017241 25080 webserver.cc:460] Webserver started at http://127.30.8.3:36103/ using document root <none> and password file <none>
I20260811 15:11:52.017912 25080 fs_manager.cc:359] Metadata directory not provided
I20260811 15:11:52.018016 25080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:11:52.022748 25080 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:11:52.025171 25101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:52.026539 25080 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:11:52.026676 25080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "e516bdb6543b44b1ba2cc2502b3316d4"
format_stamp: "Formatted at 2026-08-11 15:11:30 on dist-test-slave-1vmg"
server_key: "ad7cf47cdd29c85cd9f3eae1ea317edb"
server_key_iv: "76f282f4ca0ba0f64fdc3e3d63ca270d"
server_key_version: "encryptionkey@0"
I20260811 15:11:52.027361 25080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:11:52.045526 25080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:11:52.049831 25080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:11:52.050048 25080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:11:52.050741 25080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:11:52.051766 25080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:11:52.051833 25080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:52.051901 25080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:11:52.051931 25080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:11:52.061823 25080 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:36797
I20260811 15:11:52.061856 25320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:36797 every 8 connection(s)
I20260811 15:11:52.063133 25080 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:11:52.068243 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 25080
Aug 11 15:11:52 dist-test-slave-1vmg krb5kdc[23522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461112, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:11:52.077090 25327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:52.069015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39416 (local address 127.30.8.62:32847)
0811 15:11:52.069258 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:52.069262 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:52.069389 (+ 127us) server_negotiation.cc:407] Connection header received
0811 15:11:52.069488 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:52.069491 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:52.069539 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:52.069641 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:52.070381 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.071033 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.072180 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.072419 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.072509 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:52.073884 (+ 1375us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:52.073889 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:52.073892 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:52.073919 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:52.075425 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:52.075836 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:52.075840 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:52.075842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:52.075925 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:52.076183 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:52.076187 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:52.076189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:52.076431 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:52.076664 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:52.076781 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 15:11:52.076904 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":68,"threads_started":1}
I20260811 15:11:52.078473 24842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:52.065194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:49835 (local address 127.30.8.62:32847)
0811 15:11:52.065351 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:52.065355 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:52.066247 (+ 892us) server_negotiation.cc:407] Connection header received
0811 15:11:52.067617 (+ 1370us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:52.067620 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:52.067682 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:52.067763 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:52.069334 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.069997 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.071442 (+ 1445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.071659 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.071740 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:52.074806 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:52.074810 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:52.074812 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:52.074859 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:52.076182 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:52.076794 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:52.076797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:52.076799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:52.076894 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:52.077300 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:52.077303 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:52.077305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:52.077498 (+ 193us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:52.077593 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:52.078206 (+ 613us) server_negotiation.cc:299] Negotiation successful
0811 15:11:52.078331 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 15:11:52.079532 25325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:52.065589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:49835)
0811 15:11:52.066087 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:52.066137 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:52.067393 (+ 1256us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:52.067978 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:52.067987 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:52.068513 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:11:52.069172 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:52.069194 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.070133 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.070137 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:52.071331 (+ 1194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:52.071343 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.071815 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.071819 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:52.072029 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:52.072739 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:11:52.072762 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:11:52.074614 (+ 1852us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:11:52.076294 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:52.076302 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:52.076321 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:52.076691 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:52.076994 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:11:52.076997 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:11:52.077000 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:11:52.077213 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:11:52.077617 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:11:52.077626 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:11:52.077949 (+ 323us) client_negotiation.cc:769] Sending connection context
0811 15:11:52.078154 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 15:11:52.078498 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":118,"threads_started":1}
I20260811 15:11:52.080687 25321 heartbeater.cc:347] Connected to a master server at 127.30.8.62:32847
I20260811 15:11:52.080941 25321 heartbeater.cc:464] Registering TS with master...
I20260811 15:11:52.081496 25321 heartbeater.cc:511] Master 127.30.8.62:32847 requested a full tablet report, sending...
I20260811 15:11:52.082578 24505 ts_manager.cc:194] Registered new tserver with Master: e516bdb6543b44b1ba2cc2502b3316d4 (127.30.8.3:36797)
I20260811 15:11:52.083216 24505 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:49835
I20260811 15:11:52.091661 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 15:11:52.096598 24842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:52.092458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39422 (local address 127.30.8.62:32847)
0811 15:11:52.092618 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:52.092622 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:52.092945 (+ 323us) server_negotiation.cc:407] Connection header received
0811 15:11:52.093145 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:52.093147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:52.093279 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:52.093400 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:11:52.093700 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.094232 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.095338 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.095570 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.095654 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:52.096011 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:11:52.096211 (+ 200us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 15:11:52.096472 (+ 261us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39422: token expired
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 15:11:52.096694 24842 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39422: token expired
W20260811 15:11:52.097283 24385 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.30.8.62:32847)
I20260811 15:11:52.104825 24842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:52.097756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39438 (local address 127.30.8.62:32847)
0811 15:11:52.097867 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:52.097870 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:52.097884 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:11:52.097994 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:52.097997 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:52.098041 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:52.098113 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:11:52.098871 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.099434 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.100403 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.100603 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.100673 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:52.102020 (+ 1347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:11:52.102024 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:11:52.102026 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:11:52.102051 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:11:52.103410 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:52.103800 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:52.103802 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:52.103803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:52.103876 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:11:52.104061 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:11:52.104063 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:11:52.104064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:11:52.104306 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:11:52.104414 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:52.104588 (+ 174us) server_negotiation.cc:299] Negotiation successful
0811 15:11:52.104692 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 15:11:52.107638 24505 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39438:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:11:52.111402 24505 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:11:52.119721 24716 tablet_service.cc:1467] Processing CreateTablet for tablet a0b29f79f70748e3b475870a7f8da10b (DEFAULT_TABLE table=test-table [id=4030132d50a6469b9d98bdb7e0f221c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:11:52.120519 24716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0b29f79f70748e3b475870a7f8da10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:52.123888 24647 tablet_bootstrap.cc:492] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98: Bootstrap starting.
I20260811 15:11:52.124212 24853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:52.118536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:40879 (local address 127.0.0.1:56222)
0811 15:11:52.118905 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:52.118943 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:52.119028 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:52.120232 (+ 1204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:52.120236 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:52.120254 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:52.120349 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:52.120355 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.121797 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.121801 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:52.122846 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:52.122853 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.123802 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.123805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:52.123895 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:52.123919 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 15:11:52.123984 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 15:11:52.124071 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286}
I20260811 15:11:52.124652 25339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:52.119240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56222 (local address 127.30.8.2:40879)
0811 15:11:52.119703 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:52.119714 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:52.119751 (+ 37us) server_negotiation.cc:407] Connection header received
0811 15:11:52.119834 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:52.119840 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:52.120062 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:52.120490 (+ 428us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:52.120663 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.121680 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.122969 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.123695 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.123801 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:52.124133 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:52.124192 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 15:11:52.124521 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":142,"threads_started":1}
I20260811 15:11:52.124843 24647 tablet_bootstrap.cc:654] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98: Neither blocks nor log segments found. Creating new log.
I20260811 15:11:52.125844 25335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:52.118759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:36797 (local address 127.0.0.1:45402)
0811 15:11:52.119599 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:52.119611 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:52.119704 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:52.120193 (+ 489us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:52.120196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:52.120214 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:52.120339 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:52.120346 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.121474 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.121477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:52.122676 (+ 1199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:52.122688 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.125434 (+ 2746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.125437 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:52.125530 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:52.125555 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 15:11:52.125634 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 15:11:52.125713 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"thread_start_us":167,"threads_started":1}
I20260811 15:11:52.126194 24647 tablet_bootstrap.cc:492] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98: No bootstrap required, opened a new log
I20260811 15:11:52.126286 24647 ts_tablet_manager.cc:1397] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260811 15:11:52.126677 24647 raft_consensus.cc:348] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:11:52.126785 24647 raft_consensus.cc:374] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:52.126823 24647 raft_consensus.cc:729] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef373468f4e14efb844bde0ccc564a98, State: Initialized, Role: FOLLOWER
I20260811 15:11:52.126982 24647 consensus_queue.cc:260] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:11:52.127241 24647 ts_tablet_manager.cc:1428] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 15:11:52.127573 25338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:52.119100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45402 (local address 127.30.8.3:36797)
0811 15:11:52.119501 (+ 401us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:52.119515 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:52.119694 (+ 179us) server_negotiation.cc:407] Connection header received
0811 15:11:52.119835 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:52.119840 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:52.120062 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:52.120226 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:52.120564 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.121358 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.122816 (+ 1458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.123714 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.125445 (+ 1731us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:52.125612 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:52.125684 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 15:11:52.127423 (+ 1739us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":184,"threads_started":1}
I20260811 15:11:52.127929 24962 tablet_service.cc:1467] Processing CreateTablet for tablet a0b29f79f70748e3b475870a7f8da10b (DEFAULT_TABLE table=test-table [id=4030132d50a6469b9d98bdb7e0f221c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:11:52.129199 24962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0b29f79f70748e3b475870a7f8da10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:52.130723 25194 tablet_service.cc:1467] Processing CreateTablet for tablet a0b29f79f70748e3b475870a7f8da10b (DEFAULT_TABLE table=test-table [id=4030132d50a6469b9d98bdb7e0f221c6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:11:52.131959 25194 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0b29f79f70748e3b475870a7f8da10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:11:52.138262 25341 tablet_bootstrap.cc:492] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d: Bootstrap starting.
I20260811 15:11:52.140226 25342 tablet_bootstrap.cc:492] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: Bootstrap starting.
I20260811 15:11:52.141139 25341 tablet_bootstrap.cc:654] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d: Neither blocks nor log segments found. Creating new log.
I20260811 15:11:52.142105 25341 log.cc:824] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:52.142496 25342 tablet_bootstrap.cc:654] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: Neither blocks nor log segments found. Creating new log.
I20260811 15:11:52.143584 25342 log.cc:824] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: Log is configured to *not* fsync() on all Append() calls
I20260811 15:11:52.144381 25341 tablet_bootstrap.cc:492] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d: No bootstrap required, opened a new log
I20260811 15:11:52.144613 25341 ts_tablet_manager.cc:1397] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 15:11:52.145694 25342 tablet_bootstrap.cc:492] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: No bootstrap required, opened a new log
I20260811 15:11:52.145890 25342 ts_tablet_manager.cc:1397] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 15:11:52.148892 25341 raft_consensus.cc:348] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:11:52.149246 25341 raft_consensus.cc:374] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:52.149313 25341 raft_consensus.cc:729] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49aa3b8cd9e34767976fc8af1559c06d, State: Initialized, Role: FOLLOWER
I20260811 15:11:52.149698 25342 raft_consensus.cc:348] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:11:52.149858 25341 consensus_queue.cc:260] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:11:52.149966 25342 raft_consensus.cc:374] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:11:52.150022 25342 raft_consensus.cc:729] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e516bdb6543b44b1ba2cc2502b3316d4, State: Initialized, Role: FOLLOWER
I20260811 15:11:52.150439 25342 consensus_queue.cc:260] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:11:52.150813 25067 heartbeater.cc:503] Master 127.30.8.62:32847 was elected leader, sending a full tablet report...
I20260811 15:11:52.151099 25341 ts_tablet_manager.cc:1428] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260811 15:11:52.151553 25342 ts_tablet_manager.cc:1428] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260811 15:11:52.151686 25321 heartbeater.cc:503] Master 127.30.8.62:32847 was elected leader, sending a full tablet report...
I20260811 15:11:52.152403 25347 raft_consensus.cc:482] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:11:52.152560 25347 raft_consensus.cc:504] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:11:52.153726 25347 leader_election.cc:288] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 49aa3b8cd9e34767976fc8af1559c06d (127.30.8.2:40879), ef373468f4e14efb844bde0ccc564a98 (127.30.8.1:45981)
I20260811 15:11:52.157636 25339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:52.154311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:57245 (local address 127.30.8.2:40879)
0811 15:11:52.154424 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:52.154429 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:52.154442 (+ 13us) server_negotiation.cc:407] Connection header received
0811 15:11:52.154484 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:52.154487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:52.154524 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:52.154590 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:52.154882 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.155611 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.156619 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.157100 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.157204 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:52.157272 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:52.157442 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 15:11:52.157508 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 15:11:52.157636 25325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:52.153888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:40879 (local address 127.30.8.3:57245)
0811 15:11:52.154003 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:52.154018 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:52.154099 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:52.154617 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:52.154620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:52.154637 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:52.154754 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:52.154761 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.155712 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.155716 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:52.156499 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:52.156508 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.157217 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.157220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:52.157327 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:52.157350 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 15:11:52.157418 (+ 68us) client_negotiation.cc:240] Negotiation successful
0811 15:11:52.157498 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 15:11:52.158281 25349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:52.154160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:45981 (local address 127.30.8.3:40893)
0811 15:11:52.154987 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:52.154999 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 15:11:52.155079 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:11:52.155502 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:11:52.155504 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:11:52.155521 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:11:52.155624 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:52.155629 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.156354 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.156357 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:52.157208 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:11:52.157218 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.157921 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.157925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:11:52.158018 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:52.158042 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 15:11:52.158125 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 15:11:52.158180 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":720,"thread_start_us":122,"threads_started":1}
I20260811 15:11:52.158474 24996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0b29f79f70748e3b475870a7f8da10b" candidate_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49aa3b8cd9e34767976fc8af1559c06d" is_pre_election: true
I20260811 15:11:52.158771 25354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:52.154288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:40893 (local address 127.30.8.1:45981)
0811 15:11:52.155275 (+ 987us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:52.155279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:52.155292 (+ 13us) server_negotiation.cc:407] Connection header received
0811 15:11:52.155344 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:52.155347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:52.155390 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:52.155468 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:11:52.155740 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.156250 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.157328 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.157779 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.157886 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:52.157948 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:52.158164 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 15:11:52.158249 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":906,"thread_start_us":79,"threads_started":1}
I20260811 15:11:52.158847 24996 raft_consensus.cc:2454] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e516bdb6543b44b1ba2cc2502b3316d4 in term 0.
I20260811 15:11:52.159283 24765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0b29f79f70748e3b475870a7f8da10b" candidate_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef373468f4e14efb844bde0ccc564a98" is_pre_election: true
I20260811 15:11:52.159392 25104 leader_election.cc:302] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49aa3b8cd9e34767976fc8af1559c06d, e516bdb6543b44b1ba2cc2502b3316d4; no voters:
I20260811 15:11:52.159562 24765 raft_consensus.cc:2454] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e516bdb6543b44b1ba2cc2502b3316d4 in term 0.
I20260811 15:11:52.159644 25347 raft_consensus.cc:2781] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:11:52.159759 25347 raft_consensus.cc:482] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:11:52.159811 25347 raft_consensus.cc:3037] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:52.160938 25347 raft_consensus.cc:504] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:11:52.161283 25347 leader_election.cc:288] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [CANDIDATE]: Term 1 election: Requested vote from peers 49aa3b8cd9e34767976fc8af1559c06d (127.30.8.2:40879), ef373468f4e14efb844bde0ccc564a98 (127.30.8.1:45981)
I20260811 15:11:52.161631 24765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0b29f79f70748e3b475870a7f8da10b" candidate_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ef373468f4e14efb844bde0ccc564a98"
I20260811 15:11:52.161758 24765 raft_consensus.cc:3037] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:52.162767 24765 raft_consensus.cc:2454] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e516bdb6543b44b1ba2cc2502b3316d4 in term 1.
I20260811 15:11:52.163039 24996 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0b29f79f70748e3b475870a7f8da10b" candidate_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49aa3b8cd9e34767976fc8af1559c06d"
I20260811 15:11:52.163064 25102 leader_election.cc:302] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e516bdb6543b44b1ba2cc2502b3316d4, ef373468f4e14efb844bde0ccc564a98; no voters:
I20260811 15:11:52.163158 24996 raft_consensus.cc:3037] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:11:52.163383 25347 raft_consensus.cc:2781] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:11:52.163503 25347 raft_consensus.cc:686] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [term 1 LEADER]: Becoming Leader. State: Replica: e516bdb6543b44b1ba2cc2502b3316d4, State: Running, Role: LEADER
I20260811 15:11:52.163901 25347 consensus_queue.cc:237] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:11:52.164439 24996 raft_consensus.cc:2454] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e516bdb6543b44b1ba2cc2502b3316d4 in term 1.
I20260811 15:11:52.167611 24503 catalog_manager.cc:5495] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to e516bdb6543b44b1ba2cc2502b3316d4 (127.30.8.3). New cstate: current_term: 1 leader_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } health_report { overall_health: UNKNOWN } } }
I20260811 15:11:52.180064 25338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:52.176047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45412 (local address 127.30.8.3:36797)
0811 15:11:52.176183 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 15:11:52.176187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:11:52.176202 (+ 15us) server_negotiation.cc:407] Connection header received
0811 15:11:52.176396 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:11:52.176399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:11:52.176461 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:11:52.176570 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:11:52.176958 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.177475 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.178400 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:11:52.178621 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:11:52.178700 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:11:52.179238 (+ 538us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:11:52.179718 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:11:52.179829 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 15:11:52.179877 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 15:11:52.179936 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 15:11:52.186434 24996 raft_consensus.cc:1261] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d [term 1 FOLLOWER]: Refusing update from remote peer e516bdb6543b44b1ba2cc2502b3316d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:11:52.186429 24765 raft_consensus.cc:1261] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [term 1 FOLLOWER]: Refusing update from remote peer e516bdb6543b44b1ba2cc2502b3316d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:11:52.187215 25346 consensus_queue.cc:1035] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:11:52.187405 25347 consensus_queue.cc:1035] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:11:52.204959 24505 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:11:52.205157 24505 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39438:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:11:52.207764 24505 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: Sending DeleteTablet for 3 replicas of tablet a0b29f79f70748e3b475870a7f8da10b
I20260811 15:11:52.208856 24716 tablet_service.cc:1514] Processing DeleteTablet for tablet a0b29f79f70748e3b475870a7f8da10b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:11:52 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:58954
I20260811 15:11:52.209022 25194 tablet_service.cc:1514] Processing DeleteTablet for tablet a0b29f79f70748e3b475870a7f8da10b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:11:52 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:45402
I20260811 15:11:52.209442 24962 tablet_service.cc:1514] Processing DeleteTablet for tablet a0b29f79f70748e3b475870a7f8da10b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:11:52 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:56222
I20260811 15:11:52.209548 25379 tablet_replica.cc:330] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98: stopping tablet replica
I20260811 15:11:52.209697 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 24573
I20260811 15:11:52.209697 25379 raft_consensus.cc:2229] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:11:52.209892 25379 raft_consensus.cc:2258] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:11:52.226167 25380 tablet_replica.cc:330] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: stopping tablet replica
I20260811 15:11:52.226583 25380 raft_consensus.cc:2229] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:11:52.227130 25380 raft_consensus.cc:2258] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:11:52.227608 25381 tablet_replica.cc:330] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d: stopping tablet replica
I20260811 15:11:52.227907 25381 raft_consensus.cc:2229] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:11:52.228230 25381 raft_consensus.cc:2258] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:11:52.229694 25380 ts_tablet_manager.cc:1905] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:11:52.232357 25380 ts_tablet_manager.cc:1918] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:11:52.232462 25380 log.cc:1196] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/a0b29f79f70748e3b475870a7f8da10b
I20260811 15:11:52.232800 25380 ts_tablet_manager.cc:1939] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: Deleting consensus metadata
W20260811 15:11:52.234135 24480 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.8.1:45981 (error 108)
I20260811 15:11:52.234377 24485 catalog_manager.cc:4841] TS e516bdb6543b44b1ba2cc2502b3316d4 (127.30.8.3:36797): tablet a0b29f79f70748e3b475870a7f8da10b (table test-table [id=4030132d50a6469b9d98bdb7e0f221c6]) successfully deleted
I20260811 15:11:52.235220 25381 ts_tablet_manager.cc:1905] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:11:52.235329 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 24844
I20260811 15:11:52.238792 25335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:52.235629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:45981 (local address 127.0.0.1:58968)
0811 15:11:52.238524 (+ 2895us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:52.238659 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.1:45981: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2689}
W20260811 15:11:52.239153 24480 catalog_manager.cc:4584] TS ef373468f4e14efb844bde0ccc564a98 (127.30.8.1:45981): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a0b29f79f70748e3b475870a7f8da10b: Network error: Client connection negotiation failed: client connection to 127.30.8.1:45981: connect: Connection refused (error 111)
I20260811 15:11:52.246135 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 25080
I20260811 15:11:52.246506 25335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:11:52.246114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:40879 (local address 127.0.0.1:56224)
0811 15:11:52.246316 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 15:11:52.246391 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.2:40879: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 15:11:52.267398 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 24449
I20260811 15:12:12.288699 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:32847
--webserver_interface=127.30.8.62
--webserver_port=34339
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:42089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:32847
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:12.422889 25383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:12.423270 25383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:12.423331 25383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:12.427565 25383 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 15:12:12.427639 25383 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 15:12:12.427662 25383 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 15:12:12.427680 25383 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:12:12.427706 25383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:12.427739 25383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:12.427764 25383 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:12:12.427794 25383 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:12:12.433555 25383 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:42089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:32847
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:32847
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=34339
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:12.434870 25383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:12.436050 25383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:12.445154 25388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:12.445319 25383 server_base.cc:1034] running on GCE node
W20260811 15:12:12.445154 25389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:12.445483 25392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:12.446066 25383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:12.447317 25383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:12.448487 25383 hybrid_clock.cc:648] HybridClock initialized: now 1786461132448475 us; error 28 us; skew 500 ppm
Aug 11 15:12:12 dist-test-slave-1vmg krb5kdc[23522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:12.451671 25383 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:12:12.452755 25383 webserver.cc:460] Webserver started at http://127.30.8.62:34339/ using document root <none> and password file <none>
I20260811 15:12:12.453365 25383 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:12.453437 25383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:12.457639 25383 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 15:12:12.459985 25406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:12.461287 25383 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:12:12.461416 25383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "64fbe56d259a4dbbbcc8a4375878dcb1"
format_stamp: "Formatted at 2026-08-11 15:11:29 on dist-test-slave-1vmg"
server_key: "d672c12bbd6edc2fa7b1975f7845e1b3"
server_key_iv: "e07674c1c899f30359af879d41e5e1ff"
server_key_version: "encryptionkey@0"
I20260811 15:12:12.462018 25383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:12.499907 25383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:12.503131 25383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:12.503329 25383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:12.510783 25383 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:32847
I20260811 15:12:12.511392 25492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:32847 every 8 connection(s)
I20260811 15:12:12.512055 25383 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:12:12.516819 25493 sys_catalog.cc:263] Verifying existing consensus state
I20260811 15:12:12.518631 25493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: Bootstrap starting.
I20260811 15:12:12.519153 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 25383
I20260811 15:12:12.519577 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:45981
--local_ip_for_outbound_sockets=127.30.8.1
--tserver_master_addrs=127.30.8.62:32847
--webserver_port=41795
--webserver_interface=127.30.8.1
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:42089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:12:12.536653 25493 log.cc:824] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:12.548580 25493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 15:12:12.548995 25493 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: Bootstrap complete.
I20260811 15:12:12.553098 25493 raft_consensus.cc:348] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 32847 } }
I20260811 15:12:12.553529 25493 raft_consensus.cc:729] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 64fbe56d259a4dbbbcc8a4375878dcb1, State: Initialized, Role: FOLLOWER
I20260811 15:12:12.554060 25493 consensus_queue.cc:260] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 32847 } }
I20260811 15:12:12.554195 25493 raft_consensus.cc:388] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:12:12.554266 25493 raft_consensus.cc:482] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:12:12.554361 25493 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 2 FOLLOWER]: Advancing to term 3
I20260811 15:12:12.556439 25493 raft_consensus.cc:504] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 32847 } }
I20260811 15:12:12.556782 25493 leader_election.cc:302] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 64fbe56d259a4dbbbcc8a4375878dcb1; no voters:
I20260811 15:12:12.557108 25493 leader_election.cc:288] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 15:12:12.557220 25499 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 15:12:12.557417 25499 raft_consensus.cc:686] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [term 3 LEADER]: Becoming Leader. State: Replica: 64fbe56d259a4dbbbcc8a4375878dcb1, State: Running, Role: LEADER
I20260811 15:12:12.557744 25499 consensus_queue.cc:237] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 32847 } }
I20260811 15:12:12.558473 25493 sys_catalog.cc:564] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:12:12.558773 25499 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 64fbe56d259a4dbbbcc8a4375878dcb1. Latest consensus state: current_term: 3 leader_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 32847 } } }
I20260811 15:12:12.559034 25499 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:12.560194 25500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "64fbe56d259a4dbbbcc8a4375878dcb1" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 32847 } } }
I20260811 15:12:12.560370 25500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1 [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:12.560400 25503 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:12:12.564711 25503 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:12:12.565321 25503 catalog_manager.cc:1259] Loaded cluster ID: 5859d57c426a43058f904a91b8f67730
I20260811 15:12:12.565373 25503 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:12:12.567369 25503 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:12:12.567996 25503 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: Loaded TSK: 0
I20260811 15:12:12.568064 25503 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 64fbe56d259a4dbbbcc8a4375878dcb1: Loaded TSK: 1
F20260811 15:12:12.568675 25503 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9be5d3d9ad google::LogMessage::Fail() at ??:0
@ 0x7f9be5d4088a google::LogMessage::SendToLog() at ??:0
@ 0x7f9be5d3d4de google::LogMessage::Flush() at ??:0
@ 0x7f9be5d3ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9be8a0118d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f9be89f9f0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f9bf06a27bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f9bf067c99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f9bf067d178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f9bf06bedf8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f9bf06dd82b std::function<>::operator()() at ??:0
@ 0x7f9bf067cb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f9bf067df3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9bf067a579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f9bf06be640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f9be7f8f3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9be7f8fc83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f9be7f913f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f9be7f80512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9be95c46db start_thread at ??:0
@ 0x7f9be483b61f clone at ??:0
W20260811 15:12:12.671940 25496 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:12.672423 25496 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:12.672492 25496 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:12.678685 25496 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:12.678807 25496 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:12.678901 25496 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:12:12.687379 25496 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:42089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:45981
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=41795
--tserver_master_addrs=127.30.8.62:32847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25496
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:12.689028 25496 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:12.690483 25496 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:12.702659 25567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:12.702651 25574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:12.702657 25570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:12.703401 25496 server_base.cc:1034] running on GCE node
I20260811 15:12:12.703876 25496 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:12.704569 25496 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:12.705739 25496 hybrid_clock.cc:648] HybridClock initialized: now 1786461132705667 us; error 93 us; skew 500 ppm
Aug 11 15:12:12 dist-test-slave-1vmg krb5kdc[23522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:12.708673 25496 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:12:12.709710 25496 webserver.cc:460] Webserver started at http://127.30.8.1:41795/ using document root <none> and password file <none>
I20260811 15:12:12.710310 25496 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:12.710376 25496 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:12.714602 25496 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:12:12.716936 25587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:12.718307 25496 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 15:12:12.718420 25496 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "ef373468f4e14efb844bde0ccc564a98"
format_stamp: "Formatted at 2026-08-11 15:11:30 on dist-test-slave-1vmg"
server_key: "da47c577936bebe67f552bd72e076e67"
server_key_iv: "84ffdc38a2eda382ecaefd86b27c3b22"
server_key_version: "encryptionkey@0"
I20260811 15:12:12.719046 25496 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:12.749944 25496 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:12.753342 25496 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:12.753532 25496 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:12.754177 25496 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:12.755389 25598 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 15:12:12.759859 25496 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 15:12:12.759933 25496 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260811 15:12:12.759989 25496 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 15:12:12.761827 25496 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 15:12:12.761896 25496 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 15:12:12.762195 25598 tablet_bootstrap.cc:492] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98: Bootstrap starting.
I20260811 15:12:12.772617 25496 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:45981
I20260811 15:12:12.772951 25778 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:45981 every 8 connection(s)
I20260811 15:12:12.774020 25496 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:12:12.779120 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 25496
I20260811 15:12:12.779675 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:40879
--local_ip_for_outbound_sockets=127.30.8.2
--tserver_master_addrs=127.30.8.62:32847
--webserver_port=37559
--webserver_interface=127.30.8.2
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:42089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:12:12.783470 25598 log.cc:824] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:12.793334 25598 tablet_bootstrap.cc:492] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 15:12:12.793838 25598 tablet_bootstrap.cc:492] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98: Bootstrap complete.
I20260811 15:12:12.795218 25598 ts_tablet_manager.cc:1397] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98: Time spent bootstrapping tablet: real 0.033s user 0.021s sys 0.011s
I20260811 15:12:12.798614 25598 raft_consensus.cc:348] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:12:12.799211 25598 raft_consensus.cc:729] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ef373468f4e14efb844bde0ccc564a98, State: Initialized, Role: FOLLOWER
I20260811 15:12:12.799820 25598 consensus_queue.cc:260] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:12:12.801015 25598 ts_tablet_manager.cc:1428] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260811 15:12:12.827492 25784 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:12:12.776580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:39877)
0811 15:12:12.777245 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:12.777310 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:12.778402 (+ 1092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:12.826446 (+ 48044us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":175,"threads_started":1}
W20260811 15:12:12.828063 25779 heartbeater.cc:644] Failed to heartbeat to 127.30.8.62:32847 (0 consecutive failures): Network error: Failed to ping master at 127.30.8.62:32847: Client connection negotiation failed: client connection to 127.30.8.62:32847: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 15:12:12.828850 25784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:12.828550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:39351)
0811 15:12:12.828700 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:12.828780 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 15:12:12.829483 25784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:12.829282 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:37389)
0811 15:12:12.829378 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:12.829427 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260811 15:12:12.829650 25779 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 15:12:12.907724 25785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:12.908039 25785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:12.908085 25785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:12.912302 25785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:12.912380 25785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:12.912444 25785 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:12:12.918111 25785 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:42089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:40879
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=37559
--tserver_master_addrs=127.30.8.62:32847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:12.919365 25785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:12.920440 25785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 15:12:12.930622 25785 server_base.cc:1034] running on GCE node
W20260811 15:12:12.930568 25796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:12.930565 25793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:12.930563 25794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:12.931277 25785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:12.931990 25785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:12.933182 25785 hybrid_clock.cc:648] HybridClock initialized: now 1786461132933166 us; error 33 us; skew 500 ppm
Aug 11 15:12:12 dist-test-slave-1vmg krb5kdc[23522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:12.936347 25785 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:12:12.937361 25785 webserver.cc:460] Webserver started at http://127.30.8.2:37559/ using document root <none> and password file <none>
I20260811 15:12:12.937956 25785 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:12.938025 25785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:12.942683 25785 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:12:12.944955 25810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:12.946209 25785 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 15:12:12.946329 25785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "49aa3b8cd9e34767976fc8af1559c06d"
format_stamp: "Formatted at 2026-08-11 15:11:30 on dist-test-slave-1vmg"
server_key: "5d39b1aa107227812c5832d7b7416251"
server_key_iv: "b90becb858c2f97f41033dde0ffa9c41"
server_key_version: "encryptionkey@0"
I20260811 15:12:12.946897 25785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:12.965037 25785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:12.968328 25785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:12.968510 25785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:12.969161 25785 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:12.970316 25818 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 15:12:12.974828 25785 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 15:12:12.974906 25785 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 15:12:12.975028 25785 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 15:12:12.976854 25785 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 15:12:12.976904 25785 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 15:12:12.977176 25818 tablet_bootstrap.cc:492] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d: Bootstrap starting.
I20260811 15:12:12.983677 25818 log.cc:824] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:12.986753 25785 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:40879
I20260811 15:12:12.986797 25993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:40879 every 8 connection(s)
I20260811 15:12:12.987825 25785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:12:12.991288 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:12.989894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:55845)
0811 15:12:12.990439 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:12.990557 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":172,"threads_started":1}
W20260811 15:12:12.991698 25994 heartbeater.cc:644] Failed to heartbeat to 127.30.8.62:32847 (0 consecutive failures): Network error: Failed to ping master at 127.30.8.62:32847: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
I20260811 15:12:12.992296 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:12.992080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:53041)
0811 15:12:12.992194 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:12.992241 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260811 15:12:12.992851 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:12.992662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:58733)
0811 15:12:12.992759 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:12.992804 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260811 15:12:12.992996 25994 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 15:12:12.993112 25818 tablet_bootstrap.cc:492] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 15:12:12.993499 25818 tablet_bootstrap.cc:492] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d: Bootstrap complete.
I20260811 15:12:12.993499 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 25785
I20260811 15:12:12.993839 25818 ts_tablet_manager.cc:1397] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d: Time spent bootstrapping tablet: real 0.017s user 0.013s sys 0.000s
I20260811 15:12:12.993932 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:36797
--local_ip_for_outbound_sockets=127.30.8.3
--tserver_master_addrs=127.30.8.62:32847
--webserver_port=36103
--webserver_interface=127.30.8.3
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:42089
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:12:12.996735 25818 raft_consensus.cc:348] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:12:12.997176 25818 raft_consensus.cc:729] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49aa3b8cd9e34767976fc8af1559c06d, State: Initialized, Role: FOLLOWER
I20260811 15:12:12.997663 25818 consensus_queue.cc:260] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:12:12.998597 25818 ts_tablet_manager.cc:1428] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260811 15:12:12.999042 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:12.998673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:38309)
0811 15:12:12.998870 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:12.998956 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260811 15:12:12.999758 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:12.999508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:40031)
0811 15:12:12.999624 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:12.999678 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 15:12:13.113970 25997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:13.114295 25997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:13.114341 25997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:13.118561 25997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:13.118664 25997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:13.118726 25997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:12:13.124610 25997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:42089
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:36797
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=36103
--tserver_master_addrs=127.30.8.62:32847
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:13.125809 25997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:13.126880 25997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:13.136785 26006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:13.136829 26007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:13.136940 25997 server_base.cc:1034] running on GCE node
W20260811 15:12:13.136842 26011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:13.137519 25997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:13.138232 25997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:13.139411 25997 hybrid_clock.cc:648] HybridClock initialized: now 1786461133139393 us; error 33 us; skew 500 ppm
Aug 11 15:12:13 dist-test-slave-1vmg krb5kdc[23522](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:13.142540 25997 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:12:13.143586 25997 webserver.cc:460] Webserver started at http://127.30.8.3:36103/ using document root <none> and password file <none>
I20260811 15:12:13.144202 25997 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:13.144279 25997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:13.148411 25997 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 15:12:13.150774 26025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:13.152757 25997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:12:13.152952 25997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "e516bdb6543b44b1ba2cc2502b3316d4"
format_stamp: "Formatted at 2026-08-11 15:11:30 on dist-test-slave-1vmg"
server_key: "ad7cf47cdd29c85cd9f3eae1ea317edb"
server_key_iv: "76f282f4ca0ba0f64fdc3e3d63ca270d"
server_key_version: "encryptionkey@0"
I20260811 15:12:13.153723 25997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:13.177670 25997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:13.181154 25997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:13.181344 25997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:13.181996 25997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:13.182957 25997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:12:13.183014 25997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:13.183058 25997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:12:13.183079 25997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:13.191890 25997 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:36797
I20260811 15:12:13.191936 26225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:36797 every 8 connection(s)
I20260811 15:12:13.193032 25997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:12:13.198045 26234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:13.195398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:41573)
0811 15:12:13.196183 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:13.196997 (+ 814us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":124,"threads_started":1}
W20260811 15:12:13.198711 26229 heartbeater.cc:644] Failed to heartbeat to 127.30.8.62:32847 (0 consecutive failures): Network error: Failed to ping master at 127.30.8.62:32847: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
I20260811 15:12:13.199538 26234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:13.199275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:34623)
0811 15:12:13.199416 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:13.199474 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260811 15:12:13.200613 26234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:13.200187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:41087)
0811 15:12:13.200424 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:13.200496 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260811 15:12:13.200892 26229 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 15:12:13.202689 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 25997
I20260811 15:12:13.830897 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:13.830405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:60807)
0811 15:12:13.830708 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:13.830772 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":87,"threads_started":1}
I20260811 15:12:14.001431 26238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:14.000653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:58941)
0811 15:12:14.001139 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:14.001257 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":98,"threads_started":1}
I20260811 15:12:14.202060 26239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:14.201596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:60387)
0811 15:12:14.201880 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:14.201939 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":165,"threads_started":1}
I20260811 15:12:14.267343 26241 raft_consensus.cc:482] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:12:14.267606 26241 raft_consensus.cc:504] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:12:14.268754 26241 leader_election.cc:288] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 49aa3b8cd9e34767976fc8af1559c06d (127.30.8.2:40879), e516bdb6543b44b1ba2cc2502b3316d4 (127.30.8.3:36797)
Aug 11 15:12:14 dist-test-slave-1vmg krb5kdc[23522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.3@KRBTEST.COM
Aug 11 15:12:14 dist-test-slave-1vmg krb5kdc[23522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461132, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.2@KRBTEST.COM
I20260811 15:12:14.282593 26242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:14.269174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:39793 (local address 127.30.8.3:36797)
0811 15:12:14.269554 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:14.269583 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:14.269631 (+ 48us) server_negotiation.cc:407] Connection header received
0811 15:12:14.270380 (+ 749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:14.270387 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:14.270736 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:14.271262 (+ 526us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:14.272088 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:14.273065 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:14.274366 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:14.274839 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:14.275025 (+ 186us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:14.277750 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:14.277758 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:14.277777 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:14.277812 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:14.279647 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:14.280326 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:14.280332 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:14.280336 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:14.280455 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:14.280830 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:14.280833 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:14.280835 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:14.281323 (+ 488us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:14.281468 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:14.282087 (+ 619us) server_negotiation.cc:299] Negotiation successful
0811 15:12:14.282403 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":251,"thread_start_us":97,"threads_started":1}
I20260811 15:12:14.282578 26237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:14.269051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:40879 (local address 127.30.8.1:54797)
0811 15:12:14.269167 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:14.269182 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:14.269297 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:14.270750 (+ 1453us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:14.270756 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:14.271376 (+ 620us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:14.271921 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:14.271938 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:14.273217 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:14.273220 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:14.274247 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:14.274257 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:14.274991 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:14.274995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:14.275177 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:14.275886 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:12:14.275898 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:14.277819 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:14.279792 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:14.279795 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:14.279805 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:14.280209 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:14.280549 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:14.280552 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:14.280554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:14.280736 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:14.281756 (+ 1020us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:14.281759 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:14.281975 (+ 216us) client_negotiation.cc:769] Sending connection context
0811 15:12:14.282063 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 15:12:14.282352 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34,"mutex_wait_us":209,"spinlock_wait_cycles":27264}
I20260811 15:12:14.282882 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:14.269051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:36797 (local address 127.30.8.1:39793)
0811 15:12:14.269521 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:14.269534 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:14.269640 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:14.271000 (+ 1360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:14.271004 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:14.271378 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:14.271922 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:14.271938 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:14.273213 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:14.273215 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:14.274246 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:14.274256 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:14.274991 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:14.274995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:14.275176 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:14.275837 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:12:14.275855 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:14.277493 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:14.279782 (+ 2289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:14.279789 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:14.279804 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:14.280209 (+ 405us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:14.280550 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:14.280553 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:14.280555 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:14.280737 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:14.281453 (+ 716us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:14.281460 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:14.281715 (+ 255us) client_negotiation.cc:769] Sending connection context
0811 15:12:14.281945 (+ 230us) client_negotiation.cc:240] Negotiation successful
0811 15:12:14.282452 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":377,"spinlock_wait_cycles":65024,"thread_start_us":64,"threads_started":1}
I20260811 15:12:14.283653 26246 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:14.269343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:54797 (local address 127.30.8.2:40879)
0811 15:12:14.269621 (+ 278us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:14.269631 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:14.269665 (+ 34us) server_negotiation.cc:407] Connection header received
0811 15:12:14.269914 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:14.269919 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:14.270339 (+ 420us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:14.270955 (+ 616us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:14.272098 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:14.273059 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:14.274370 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:14.274838 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:14.275043 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:14.277995 (+ 2952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:14.278002 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:14.278018 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:14.278053 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:14.279678 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:14.280326 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:14.280332 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:14.280336 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:14.280455 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:14.280830 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:14.280832 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:14.280833 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:14.281303 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:14.283114 (+ 1811us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:14.283242 (+ 128us) server_negotiation.cc:299] Negotiation successful
0811 15:12:14.283477 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":90,"threads_started":1}
I20260811 15:12:14.284687 25915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0b29f79f70748e3b475870a7f8da10b" candidate_uuid: "ef373468f4e14efb844bde0ccc564a98" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "49aa3b8cd9e34767976fc8af1559c06d" is_pre_election: true
I20260811 15:12:14.285001 25915 raft_consensus.cc:2454] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ef373468f4e14efb844bde0ccc564a98 in term 1.
W20260811 15:12:14.285398 25593 leader_election.cc:341] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer e516bdb6543b44b1ba2cc2502b3316d4 (127.30.8.3:36797): Not found: Tablet not found: a0b29f79f70748e3b475870a7f8da10b
I20260811 15:12:14.285677 25592 leader_election.cc:302] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49aa3b8cd9e34767976fc8af1559c06d, ef373468f4e14efb844bde0ccc564a98; no voters: e516bdb6543b44b1ba2cc2502b3316d4
I20260811 15:12:14.285921 26241 raft_consensus.cc:2781] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 15:12:14.286002 26241 raft_consensus.cc:482] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:12:14.286041 26241 raft_consensus.cc:3037] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [term 1 FOLLOWER]: Advancing to term 2
I20260811 15:12:14.288280 26241 raft_consensus.cc:504] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:12:14.288661 26241 leader_election.cc:288] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [CANDIDATE]: Term 2 election: Requested vote from peers 49aa3b8cd9e34767976fc8af1559c06d (127.30.8.2:40879), e516bdb6543b44b1ba2cc2502b3316d4 (127.30.8.3:36797)
I20260811 15:12:14.289096 25915 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a0b29f79f70748e3b475870a7f8da10b" candidate_uuid: "ef373468f4e14efb844bde0ccc564a98" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "49aa3b8cd9e34767976fc8af1559c06d"
I20260811 15:12:14.289229 25915 raft_consensus.cc:3037] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d [term 1 FOLLOWER]: Advancing to term 2
W20260811 15:12:14.289384 25593 leader_election.cc:341] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer e516bdb6543b44b1ba2cc2502b3316d4 (127.30.8.3:36797): Not found: Tablet not found: a0b29f79f70748e3b475870a7f8da10b
I20260811 15:12:14.291486 25915 raft_consensus.cc:2454] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ef373468f4e14efb844bde0ccc564a98 in term 2.
I20260811 15:12:14.291810 25592 leader_election.cc:302] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 49aa3b8cd9e34767976fc8af1559c06d, ef373468f4e14efb844bde0ccc564a98; no voters: e516bdb6543b44b1ba2cc2502b3316d4
I20260811 15:12:14.291998 26241 raft_consensus.cc:2781] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 15:12:14.292227 26241 raft_consensus.cc:686] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [term 2 LEADER]: Becoming Leader. State: Replica: ef373468f4e14efb844bde0ccc564a98, State: Running, Role: LEADER
I20260811 15:12:14.292629 26241 consensus_queue.cc:237] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:12:14.293087 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:14.292841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:38217)
0811 15:12:14.292964 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:14.293015 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 15:12:14.702483 25915 raft_consensus.cc:1261] T a0b29f79f70748e3b475870a7f8da10b P 49aa3b8cd9e34767976fc8af1559c06d [term 2 FOLLOWER]: Refusing update from remote peer ef373468f4e14efb844bde0ccc564a98: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260811 15:12:14.703289 26241 consensus_queue.cc:1035] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260811 15:12:14.705574 26250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:14.705168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:56305)
0811 15:12:14.705378 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:14.705451 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":58,"threads_started":1}
I20260811 15:12:14.708473 26250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:14.708166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:37823)
0811 15:12:14.708335 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:14.708388 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260811 15:12:14.708575 26245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:14.707986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:44919)
0811 15:12:14.708424 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:14.708481 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366}
W20260811 15:12:14.708690 25593 consensus_peers.cc:487] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98 -> Peer e516bdb6543b44b1ba2cc2502b3316d4 (127.30.8.3:36797): Couldn't send request to peer e516bdb6543b44b1ba2cc2502b3316d4. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: a0b29f79f70748e3b475870a7f8da10b. This is attempt 1: this message will repeat every 5th retry.
I20260811 15:12:15.203450 26254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:15.202973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:55801)
0811 15:12:15.203263 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:15.203321 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":208,"thread_start_us":109,"threads_started":1}
I20260811 15:12:15.283396 26257 ts_tablet_manager.cc:927] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: Initiating tablet copy from peer ef373468f4e14efb844bde0ccc564a98 (127.30.8.1:45981)
I20260811 15:12:15.284090 26257 tablet_copy_client.cc:316] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: tablet copy: Beginning tablet copy session from remote peer at address 127.30.8.1:45981
Aug 11 15:12:15 dist-test-slave-1vmg krb5kdc[23522](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461133, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.1@KRBTEST.COM
I20260811 15:12:15.293995 26254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:15.284584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:45981 (local address 127.30.8.3:34289)
0811 15:12:15.284709 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:15.284728 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:15.284871 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:15.285414 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:15.285421 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:15.285814 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:15.285928 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:15.285934 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:15.286893 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:15.286897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:15.287539 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:15.287548 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:15.288136 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:15.288140 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:15.288250 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:15.288828 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:12:15.288841 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:15.290258 (+ 1417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:15.292163 (+ 1905us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:15.292173 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:15.292180 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:15.292412 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:15.292707 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:15.292710 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:15.292712 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:15.292791 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:15.293325 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:15.293333 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:15.293470 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 15:12:15.293607 (+ 137us) client_negotiation.cc:240] Negotiation successful
0811 15:12:15.293805 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
I20260811 15:12:15.294005 26261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:15.284725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:34289 (local address 127.30.8.1:45981)
0811 15:12:15.284992 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:15.285000 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:15.285022 (+ 22us) server_negotiation.cc:407] Connection header received
0811 15:12:15.285063 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:15.285068 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:15.285262 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:15.285388 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:15.286054 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:15.286782 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:15.287671 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:15.288022 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:15.288120 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:15.290397 (+ 2277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:15.290407 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:15.290421 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:15.290446 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:15.292035 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:15.292528 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:15.292533 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:15.292534 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:15.292618 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:15.292887 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:15.292890 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:15.292892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:15.293207 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:15.293347 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:15.293707 (+ 360us) server_negotiation.cc:299] Negotiation successful
0811 15:12:15.293861 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":68,"threads_started":1}
I20260811 15:12:15.295286 25737 tablet_copy_service.cc:140] P ef373468f4e14efb844bde0ccc564a98: Received BeginTabletCopySession request for tablet a0b29f79f70748e3b475870a7f8da10b from peer e516bdb6543b44b1ba2cc2502b3316d4 ({username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:34289)
I20260811 15:12:15.295454 25737 tablet_copy_service.cc:161] P ef373468f4e14efb844bde0ccc564a98: Beginning new tablet copy session on tablet a0b29f79f70748e3b475870a7f8da10b from peer e516bdb6543b44b1ba2cc2502b3316d4 at {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:34289: session id = e516bdb6543b44b1ba2cc2502b3316d4-a0b29f79f70748e3b475870a7f8da10b
I20260811 15:12:15.296648 25737 tablet_copy_source_session.cc:205] T a0b29f79f70748e3b475870a7f8da10b P ef373468f4e14efb844bde0ccc564a98: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 15:12:15.300971 26257 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a0b29f79f70748e3b475870a7f8da10b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:15.307907 26257 tablet_copy_client.cc:791] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: tablet copy: Starting download of 0 data blocks...
I20260811 15:12:15.308121 26257 tablet_copy_client.cc:655] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: tablet copy: Starting download of 1 WAL segments...
I20260811 15:12:15.309887 26257 tablet_copy_client.cc:531] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 15:12:15.311805 26257 tablet_bootstrap.cc:492] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: Bootstrap starting.
I20260811 15:12:15.317473 26257 log.cc:824] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:15.324661 26257 tablet_bootstrap.cc:492] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 15:12:15.325047 26257 tablet_bootstrap.cc:492] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: Bootstrap complete.
I20260811 15:12:15.325429 26257 ts_tablet_manager.cc:1397] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260811 15:12:15.328105 26257 raft_consensus.cc:348] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:12:15.328369 26257 raft_consensus.cc:729] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: e516bdb6543b44b1ba2cc2502b3316d4, State: Initialized, Role: FOLLOWER
I20260811 15:12:15.328877 26257 consensus_queue.cc:260] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "49aa3b8cd9e34767976fc8af1559c06d" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40879 } } peers { permanent_uuid: "e516bdb6543b44b1ba2cc2502b3316d4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 36797 } } peers { permanent_uuid: "ef373468f4e14efb844bde0ccc564a98" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45981 } }
I20260811 15:12:15.330013 26257 ts_tablet_manager.cc:1428] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260811 15:12:15.330335 26254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:15.330015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:60425)
0811 15:12:15.330188 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:15.330252 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260811 15:12:15.330972 25737 tablet_copy_service.cc:342] P ef373468f4e14efb844bde0ccc564a98: Request end of tablet copy session e516bdb6543b44b1ba2cc2502b3316d4-a0b29f79f70748e3b475870a7f8da10b received from {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:34289
I20260811 15:12:15.331095 25737 tablet_copy_service.cc:433] P ef373468f4e14efb844bde0ccc564a98: ending tablet copy session e516bdb6543b44b1ba2cc2502b3316d4-a0b29f79f70748e3b475870a7f8da10b on tablet a0b29f79f70748e3b475870a7f8da10b with peer e516bdb6543b44b1ba2cc2502b3316d4
I20260811 15:12:15.331243 26254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:15.330997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:60261)
0811 15:12:15.331125 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:15.331177 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260811 15:12:15.703404 26269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:15.702972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:57789)
0811 15:12:15.703250 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:15.703302 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":79,"threads_started":1}
I20260811 15:12:15.703382 26149 raft_consensus.cc:1203] T a0b29f79f70748e3b475870a7f8da10b P e516bdb6543b44b1ba2cc2502b3316d4 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260811 15:12:15.704360 26254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:15.704110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:44541)
0811 15:12:15.704221 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:15.704282 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 15:12:15.709482 26270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:15.709116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:57771)
0811 15:12:15.709340 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:15.709391 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":70,"threads_started":1}
I20260811 15:12:16.704931 26271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:16.704366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:37929)
0811 15:12:16.704705 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:16.704772 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":94,"threads_started":1}
I20260811 15:12:16.705540 26272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:16.705070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:58341)
0811 15:12:16.705391 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:16.705452 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":70,"threads_started":1}
I20260811 15:12:16.710790 26273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:16.710290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:36149)
0811 15:12:16.710631 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:16.710690 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":97,"threads_started":1}
I20260811 15:12:17.706388 26274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:17.705979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:40029)
0811 15:12:17.706242 (+ 263us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:17.706292 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":100,"threads_started":1}
I20260811 15:12:17.706557 26276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:17.706180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:32785)
0811 15:12:17.706404 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:17.706455 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":69,"threads_started":1}
I20260811 15:12:17.712491 26277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:17.711908 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:55089)
0811 15:12:17.712273 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:17.712357 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":119,"threads_started":1}
I20260811 15:12:18.707988 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:18.707549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:53713)
0811 15:12:18.707828 (+ 279us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:18.707887 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":82,"threads_started":1}
I20260811 15:12:18.707988 26281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:18.707584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:58299)
0811 15:12:18.707831 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:18.707887 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":75,"threads_started":1}
I20260811 15:12:18.708678 26280 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:18.708441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:56799)
0811 15:12:18.708552 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:18.708601 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260811 15:12:18.714219 26282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:18.713670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:45621)
0811 15:12:18.714033 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:18.714114 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":149,"threads_started":1}
I20260811 15:12:19.709422 26287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:19.708905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:50757)
0811 15:12:19.709248 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:19.709318 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":86,"threads_started":1}
I20260811 15:12:19.709774 26289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:19.709389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:41601)
0811 15:12:19.709629 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:19.709690 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":71,"threads_started":1}
I20260811 15:12:19.715711 26290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:19.715108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:57711)
0811 15:12:19.715490 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:19.715574 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":112,"threads_started":1}
I20260811 15:12:20.711043 26295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:20.710495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:51133)
0811 15:12:20.710839 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:20.710895 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":273,"thread_start_us":96,"threads_started":1}
I20260811 15:12:20.711133 26292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:20.710453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:37263)
0811 15:12:20.710762 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:20.710826 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":105,"threads_started":1}
I20260811 15:12:20.717342 26296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:20.716688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:48891)
0811 15:12:20.717102 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:20.717204 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":103,"threads_started":1}
I20260811 15:12:21.712798 26301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:21.712280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:40171)
0811 15:12:21.712611 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:21.712674 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":147,"threads_started":1}
I20260811 15:12:21.712798 26299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:21.712189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:37141)
0811 15:12:21.712596 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:21.712674 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":119,"threads_started":1}
I20260811 15:12:21.718647 26304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:21.718205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:58707)
0811 15:12:21.718478 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:21.718540 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":88,"threads_started":1}
I20260811 15:12:22.714299 26306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:22.713713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:54017)
0811 15:12:22.714049 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:22.714132 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":90,"threads_started":1}
I20260811 15:12:22.714351 26308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:22.713778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:52751)
0811 15:12:22.714147 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:22.714218 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":119,"threads_started":1}
I20260811 15:12:22.720120 26309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:22.719527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:58905)
0811 15:12:22.719902 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:22.719990 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":100,"threads_started":1}
I20260811 15:12:23.715818 26310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:23.715404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:36023)
0811 15:12:23.715660 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:23.715719 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":83,"threads_started":1}
I20260811 15:12:23.716174 26311 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:23.715779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:50815)
0811 15:12:23.716022 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:23.716081 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":77,"threads_started":1}
I20260811 15:12:23.721608 26313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:23.721181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:38391)
0811 15:12:23.721442 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:23.721501 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":96,"threads_started":1}
I20260811 15:12:24.717278 26316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:24.716836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:42305)
0811 15:12:24.717112 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:24.717171 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":87,"threads_started":1}
I20260811 15:12:24.717307 26314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:24.716739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:59511)
0811 15:12:24.717117 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:24.717178 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":121,"threads_started":1}
I20260811 15:12:24.722967 26317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:24.722443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:47637)
0811 15:12:24.722742 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:24.722807 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":111,"threads_started":1}
I20260811 15:12:25.718771 26319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:25.718193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:35079)
0811 15:12:25.718571 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:25.718659 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":141,"threads_started":1}
I20260811 15:12:25.718839 26320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:25.718281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:35999)
0811 15:12:25.718654 (+ 373us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:25.718716 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":96,"threads_started":1}
I20260811 15:12:25.724215 26327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:25.723767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:41569)
0811 15:12:25.724040 (+ 273us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:25.724109 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":101,"threads_started":1}
I20260811 15:12:26.720467 26329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:26.719869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:60673)
0811 15:12:26.720261 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:26.720332 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":114,"threads_started":1}
I20260811 15:12:26.720459 26328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:26.719869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:52293)
0811 15:12:26.720206 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:26.720308 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":109,"threads_started":1}
I20260811 15:12:26.725512 26330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:26.725031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:41823)
0811 15:12:26.725356 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:26.725409 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":80,"threads_started":1}
I20260811 15:12:27.721930 26331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:27.721458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.3:42007)
0811 15:12:27.721755 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:27.721824 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":83,"threads_started":1}
I20260811 15:12:27.722016 26335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:27.721559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.1:39229)
0811 15:12:27.721843 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:27.721898 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":90,"threads_started":1}
I20260811 15:12:27.726717 26336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:27.726264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:32847 (local address 127.30.8.2:40663)
0811 15:12:27.726529 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:27.726596 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.62:32847: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":76,"threads_started":1}
I20260811 15:12:28.210476 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 25496
I20260811 15:12:28.226049 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 25785
I20260811 15:12:28.242439 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 25997
2026-08-11T15:12:28Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59835 ms)
[----------] 30 tests from SecurityITest (254689 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:12:28 dist-test-slave-1vmg krb5kdc[26340](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:12:28 dist-test-slave-1vmg krb5kdc[26340](info): set up 2 sockets
Aug 11 15:12:28 dist-test-slave-1vmg krb5kdc[26340](info): commencing operation
krb5kdc: starting...
W20260811 15:12:29.439137 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:12:29 dist-test-slave-1vmg krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:12:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:12:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:12:29.565299 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:40381
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:36309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:40381
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:29.680542 26363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:29.680851 26363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:29.680893 26363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:29.684944 26363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:12:29.685020 26363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:29.685039 26363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:29.685055 26363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:12:29.685075 26363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:12:29.690593 26363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:40381
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:40381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:29.691790 26363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:29.692811 26363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:29.701546 26371 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:29.701601 26363 server_base.cc:1034] running on GCE node
W20260811 15:12:29.701593 26372 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:29.701854 26376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:29.702488 26363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:29.703713 26363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:29.704893 26363 hybrid_clock.cc:648] HybridClock initialized: now 1786461149704876 us; error 29 us; skew 500 ppm
Aug 11 15:12:29 dist-test-slave-1vmg krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461149, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:29.707713 26363 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:12:29.708653 26363 webserver.cc:460] Webserver started at http://127.30.8.62:39559/ using document root <none> and password file <none>
I20260811 15:12:29.709226 26363 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:29.709298 26363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:29.709520 26363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:29.711731 26363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "74bccfe3ab314f989dd19610ad57f8a0"
format_stamp: "Formatted at 2026-08-11 15:12:29 on dist-test-slave-1vmg"
server_key: "77b75b389ef6e8d25896695f4c15ca9d"
server_key_iv: "f084e50c13b27421991ce532e2b13217"
server_key_version: "encryptionkey@0"
I20260811 15:12:29.712298 26363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "74bccfe3ab314f989dd19610ad57f8a0"
format_stamp: "Formatted at 2026-08-11 15:12:29 on dist-test-slave-1vmg"
server_key: "77b75b389ef6e8d25896695f4c15ca9d"
server_key_iv: "f084e50c13b27421991ce532e2b13217"
server_key_version: "encryptionkey@0"
I20260811 15:12:29.716228 26363 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:12:29.718647 26392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:29.719983 26363 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:12:29.720144 26363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "74bccfe3ab314f989dd19610ad57f8a0"
format_stamp: "Formatted at 2026-08-11 15:12:29 on dist-test-slave-1vmg"
server_key: "77b75b389ef6e8d25896695f4c15ca9d"
server_key_iv: "f084e50c13b27421991ce532e2b13217"
server_key_version: "encryptionkey@0"
I20260811 15:12:29.720257 26363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:29.733705 26363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:29.736616 26363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:29.736810 26363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:29.743891 26363 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:40381
I20260811 15:12:29.743912 26479 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:40381 every 8 connection(s)
I20260811 15:12:29.744942 26363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:12:29.747879 26480 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:29.752277 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 26363
I20260811 15:12:29.752394 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:12:29.752660 30752 external_mini_cluster.cc:1428] Setting key 5d9d7112b4dcc2f872bc4375663fe0b7
I20260811 15:12:29.754048 26480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0: Bootstrap starting.
I20260811 15:12:29.756916 26480 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:29.757671 26480 log.cc:824] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:29.759672 26480 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0: No bootstrap required, opened a new log
Aug 11 15:12:29 dist-test-slave-1vmg krb5kdc[26340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:12:29.763598 26480 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74bccfe3ab314f989dd19610ad57f8a0" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 40381 } }
I20260811 15:12:29.763849 26480 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:29.763908 26480 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74bccfe3ab314f989dd19610ad57f8a0, State: Initialized, Role: FOLLOWER
I20260811 15:12:29.764410 26480 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74bccfe3ab314f989dd19610ad57f8a0" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 40381 } }
I20260811 15:12:29.764582 26480 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:12:29.764672 26480 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:12:29.764777 26480 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:29.765987 26480 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74bccfe3ab314f989dd19610ad57f8a0" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 40381 } }
I20260811 15:12:29.766369 26480 leader_election.cc:302] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74bccfe3ab314f989dd19610ad57f8a0; no voters:
I20260811 15:12:29.766736 26480 leader_election.cc:288] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:12:29.766839 26486 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:29.767066 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:29.754170 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46540 (local address 127.30.8.62:40381)
0811 15:12:29.754617 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:29.754636 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:29.754681 (+ 45us) server_negotiation.cc:407] Connection header received
0811 15:12:29.755573 (+ 892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:29.755598 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:29.756022 (+ 424us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:29.756394 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:29.757083 (+ 689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:29.758121 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:29.758963 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:29.759440 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:29.759602 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:29.762025 (+ 2423us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:29.762031 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:29.762045 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:29.762073 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:29.763729 (+ 1656us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:29.764217 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:29.764223 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:29.764229 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:29.764378 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:29.764625 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:29.764628 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:29.764630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:29.765201 (+ 571us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:29.765388 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:29.765615 (+ 227us) server_negotiation.cc:299] Negotiation successful
0811 15:12:29.765949 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":121,"threads_started":1}
I20260811 15:12:29.767071 26486 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0 [term 1 LEADER]: Becoming Leader. State: Replica: 74bccfe3ab314f989dd19610ad57f8a0, State: Running, Role: LEADER
I20260811 15:12:29.767776 26486 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74bccfe3ab314f989dd19610ad57f8a0" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 40381 } }
I20260811 15:12:29.768100 26480 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:12:29.770833 26488 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74bccfe3ab314f989dd19610ad57f8a0. Latest consensus state: current_term: 1 leader_uuid: "74bccfe3ab314f989dd19610ad57f8a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74bccfe3ab314f989dd19610ad57f8a0" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 40381 } } }
I20260811 15:12:29.770956 26487 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74bccfe3ab314f989dd19610ad57f8a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74bccfe3ab314f989dd19610ad57f8a0" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 40381 } } }
I20260811 15:12:29.771109 26488 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0 [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:29.771167 26487 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0 [sys.catalog]: This master's current role is: LEADER
W20260811 15:12:29.773454 26510 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:12:29.773521 26510 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:12:29.773599 26512 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:12:29.774315 26512 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:12:29.779645 26512 catalog_manager.cc:1347] Generated new cluster ID: 9515a08ff83d4e89822544b9ed6950b1
I20260811 15:12:29.779738 26512 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:12:29.798048 26512 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:12:29.799489 26512 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:12:29.815907 26512 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0: Generated new TSK 0
I20260811 15:12:29.816874 26512 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
I20260811 15:12:29.853199 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:40381
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:36309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:29.969594 26520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:29.969879 26520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:29.969924 26520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:29.973954 26520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:29.974036 26520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:29.974114 26520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:12:29.979465 26520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:40381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:29.980631 26520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:29.981626 26520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:29.991204 26533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:29.991318 26529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:29.991197 26530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:29.991603 26520 server_base.cc:1034] running on GCE node
I20260811 15:12:29.992020 26520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:29.992690 26520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:29.993848 26520 hybrid_clock.cc:648] HybridClock initialized: now 1786461149993822 us; error 42 us; skew 500 ppm
Aug 11 15:12:29 dist-test-slave-1vmg krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461149, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:29.996574 26520 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:12:29.997455 26520 webserver.cc:460] Webserver started at http://127.30.8.1:33455/ using document root <none> and password file <none>
I20260811 15:12:29.998027 26520 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:29.998097 26520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:29.998308 26520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:30.000444 26520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "6059fc590be24071a353ec55ae1b0665"
format_stamp: "Formatted at 2026-08-11 15:12:29 on dist-test-slave-1vmg"
server_key: "78faf5e1578f304c6f8b388ceaae48fb"
server_key_iv: "ec3b90f3b274f60f7a53903cf94b3cab"
server_key_version: "encryptionkey@0"
I20260811 15:12:30.000993 26520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "6059fc590be24071a353ec55ae1b0665"
format_stamp: "Formatted at 2026-08-11 15:12:29 on dist-test-slave-1vmg"
server_key: "78faf5e1578f304c6f8b388ceaae48fb"
server_key_iv: "ec3b90f3b274f60f7a53903cf94b3cab"
server_key_version: "encryptionkey@0"
I20260811 15:12:30.004789 26520 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 15:12:30.007187 26548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:30.008370 26520 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 15:12:30.008488 26520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "6059fc590be24071a353ec55ae1b0665"
format_stamp: "Formatted at 2026-08-11 15:12:29 on dist-test-slave-1vmg"
server_key: "78faf5e1578f304c6f8b388ceaae48fb"
server_key_iv: "ec3b90f3b274f60f7a53903cf94b3cab"
server_key_version: "encryptionkey@0"
I20260811 15:12:30.008603 26520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:30.032773 26520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:30.035687 26520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:30.035883 26520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:30.036491 26520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:30.037361 26520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:12:30.037411 26520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:30.037465 26520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:12:30.037492 26520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:30.045950 26520 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:39053
I20260811 15:12:30.046001 26741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:39053 every 8 connection(s)
I20260811 15:12:30.046862 26520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:12:30.050952 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 26520
I20260811 15:12:30.051085 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:12:30.051357 30752 external_mini_cluster.cc:1428] Setting key 52d0dfcb7da51a6645a112a6c08462d1
Aug 11 15:12:30 dist-test-slave-1vmg krb5kdc[26340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461149, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:12:30.060510 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:30.048680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:52529 (local address 127.30.8.62:40381)
0811 15:12:30.048810 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:30.048813 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:30.049859 (+ 1046us) server_negotiation.cc:407] Connection header received
0811 15:12:30.050732 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:30.050735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:30.050793 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:30.050872 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:30.052237 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.052827 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.054038 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.054264 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.054353 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:30.057043 (+ 2690us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:30.057046 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:30.057048 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:30.057073 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:30.058426 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:30.058910 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:30.058937 (+ 27us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:30.058939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:30.059027 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:30.059361 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:30.059364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:30.059366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:30.059568 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:30.059656 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:30.060196 (+ 540us) server_negotiation.cc:299] Negotiation successful
0811 15:12:30.060335 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 15:12:30.061357 26748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:30.049013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:40381 (local address 127.30.8.1:52529)
0811 15:12:30.049707 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:30.049749 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:30.050528 (+ 779us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:30.051099 (+ 571us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:30.051108 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:30.051517 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:30.052101 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:30.052119 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.052949 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.052952 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:30.053917 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:30.053927 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.054406 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.054410 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:30.054611 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:30.055251 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:12:30.055272 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:30.056874 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:30.058535 (+ 1661us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:30.058541 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:30.058559 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:30.058812 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:30.059112 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:30.059115 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:30.059117 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:30.059270 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:30.059667 (+ 397us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:30.059674 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:30.059949 (+ 275us) client_negotiation.cc:769] Sending connection context
0811 15:12:30.060137 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 15:12:30.060482 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":489,"thread_start_us":143,"threads_started":1}
WARNING: no policy specified for kudu/127.30.8.2@KRBTEST.COM; defaulting to no policy
I20260811 15:12:30.062616 26743 heartbeater.cc:347] Connected to a master server at 127.30.8.62:40381
I20260811 15:12:30.062885 26743 heartbeater.cc:464] Registering TS with master...
I20260811 15:12:30.063445 26743 heartbeater.cc:511] Master 127.30.8.62:40381 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.2@KRBTEST.COM" created.
I20260811 15:12:30.065054 26421 ts_manager.cc:194] Registered new tserver with Master: 6059fc590be24071a353ec55ae1b0665 (127.30.8.1:39053)
I20260811 15:12:30.066170 26421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:52529
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
I20260811 15:12:30.079334 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:40381
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:36309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:30.193774 26751 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:30.194065 26751 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:30.194109 26751 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:30.198066 26751 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:30.198134 26751 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:30.198194 26751 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:12:30.203573 26751 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:40381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26751
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:30.204672 26751 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:30.205658 26751 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 15:12:30.215319 26751 server_base.cc:1034] running on GCE node
W20260811 15:12:30.215281 26764 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:30.215281 26765 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:30.215667 26769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:30.216087 26751 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:30.216698 26751 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:30.217852 26751 hybrid_clock.cc:648] HybridClock initialized: now 1786461150217822 us; error 39 us; skew 500 ppm
Aug 11 15:12:30 dist-test-slave-1vmg krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461150, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:30.220621 26751 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:12:30.221529 26751 webserver.cc:460] Webserver started at http://127.30.8.2:40507/ using document root <none> and password file <none>
I20260811 15:12:30.222095 26751 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:30.222159 26751 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:30.222354 26751 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:30.224557 26751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "0372f82447b14b9babbf379e01f1ccd8"
format_stamp: "Formatted at 2026-08-11 15:12:30 on dist-test-slave-1vmg"
server_key: "9e6caede4cfe1c472ae9e445e34d9ca9"
server_key_iv: "d2b49e1dfc3cb2e1d73070885cba455a"
server_key_version: "encryptionkey@0"
I20260811 15:12:30.225167 26751 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "0372f82447b14b9babbf379e01f1ccd8"
format_stamp: "Formatted at 2026-08-11 15:12:30 on dist-test-slave-1vmg"
server_key: "9e6caede4cfe1c472ae9e445e34d9ca9"
server_key_iv: "d2b49e1dfc3cb2e1d73070885cba455a"
server_key_version: "encryptionkey@0"
I20260811 15:12:30.229017 26751 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:12:30.231493 26780 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:30.232702 26751 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:12:30.232818 26751 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "0372f82447b14b9babbf379e01f1ccd8"
format_stamp: "Formatted at 2026-08-11 15:12:30 on dist-test-slave-1vmg"
server_key: "9e6caede4cfe1c472ae9e445e34d9ca9"
server_key_iv: "d2b49e1dfc3cb2e1d73070885cba455a"
server_key_version: "encryptionkey@0"
I20260811 15:12:30.232918 26751 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:30.247148 26751 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:30.250380 26751 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:30.250571 26751 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:30.251245 26751 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:30.252104 26751 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:12:30.252166 26751 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:30.252226 26751 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:12:30.252250 26751 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:30.261023 26751 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:40713
I20260811 15:12:30.261051 26961 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:40713 every 8 connection(s)
I20260811 15:12:30.261930 26751 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:12:30.266484 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 26751
I20260811 15:12:30.266618 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:12:30.266875 30752 external_mini_cluster.cc:1428] Setting key b44684f466d4366d00c3ce6fc967b683
Aug 11 15:12:30 dist-test-slave-1vmg krb5kdc[26340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461150, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:12:30.276355 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:30.263817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:48219 (local address 127.30.8.62:40381)
0811 15:12:30.263987 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:30.263992 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:30.264817 (+ 825us) server_negotiation.cc:407] Connection header received
0811 15:12:30.266043 (+ 1226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:30.266046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:30.266106 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:30.266182 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:30.267498 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.268108 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.269676 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.269953 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.270040 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:30.272637 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:30.272641 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:30.272643 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:30.272671 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:30.274210 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:30.274705 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:30.274708 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:30.274710 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:30.274806 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:30.275173 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:30.275176 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:30.275178 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:30.275420 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:30.275527 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:30.276038 (+ 511us) server_negotiation.cc:299] Negotiation successful
0811 15:12:30.276177 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 15:12:30.277118 26966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:30.264214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:40381 (local address 127.30.8.2:48219)
0811 15:12:30.264668 (+ 454us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:30.264711 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:30.265831 (+ 1120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:30.266359 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:30.266365 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:30.266768 (+ 403us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:30.267339 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:30.267357 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.268233 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.268237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:30.269560 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:30.269571 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.270100 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.270104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:30.270298 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:30.270986 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:12:30.271006 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:30.272467 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:30.274329 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:30.274335 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:30.274349 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:30.274607 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:30.274887 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:30.274889 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:30.274890 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:30.275081 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:30.275511 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:30.275517 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:30.275773 (+ 256us) client_negotiation.cc:769] Sending connection context
0811 15:12:30.275934 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 15:12:30.276219 (+ 285us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":117,"threads_started":1}
WARNING: no policy specified for kudu/127.30.8.3@KRBTEST.COM; defaulting to no policy
I20260811 15:12:30.278339 26962 heartbeater.cc:347] Connected to a master server at 127.30.8.62:40381
I20260811 15:12:30.278589 26962 heartbeater.cc:464] Registering TS with master...
I20260811 15:12:30.279145 26962 heartbeater.cc:511] Master 127.30.8.62:40381 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.3@KRBTEST.COM" created.
I20260811 15:12:30.280277 26421 ts_manager.cc:194] Registered new tserver with Master: 0372f82447b14b9babbf379e01f1ccd8 (127.30.8.2:40713)
I20260811 15:12:30.280840 26421 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:48219
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
I20260811 15:12:30.295742 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:40381
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:36309
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:30.410493 26971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:30.410799 26971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:30.410847 26971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:30.414997 26971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:30.415074 26971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:30.415136 26971 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:12:30.420581 26971 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36309
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:40381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:30.421725 26971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:30.422747 26971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:30.432430 26985 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:30.432639 26971 server_base.cc:1034] running on GCE node
W20260811 15:12:30.432451 26988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:30.432466 26980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:30.433238 26971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:30.433924 26971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:30.435107 26971 hybrid_clock.cc:648] HybridClock initialized: now 1786461150435081 us; error 41 us; skew 500 ppm
Aug 11 15:12:30 dist-test-slave-1vmg krb5kdc[26340](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461150, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:30.437887 26971 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:12:30.438796 26971 webserver.cc:460] Webserver started at http://127.30.8.3:40399/ using document root <none> and password file <none>
I20260811 15:12:30.439402 26971 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:30.439474 26971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:30.439698 26971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:30.441793 26971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "7bd1b569917d417bb2c1ed25b7d974b4"
format_stamp: "Formatted at 2026-08-11 15:12:30 on dist-test-slave-1vmg"
server_key: "21d7c3d0b532f4c572dbd39a292f1f70"
server_key_iv: "9335fa93d679810b3dfffb571f90c535"
server_key_version: "encryptionkey@0"
I20260811 15:12:30.442410 26971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "7bd1b569917d417bb2c1ed25b7d974b4"
format_stamp: "Formatted at 2026-08-11 15:12:30 on dist-test-slave-1vmg"
server_key: "21d7c3d0b532f4c572dbd39a292f1f70"
server_key_iv: "9335fa93d679810b3dfffb571f90c535"
server_key_version: "encryptionkey@0"
I20260811 15:12:30.446174 26971 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 15:12:30.448688 27000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:30.449885 26971 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 15:12:30.450002 26971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "7bd1b569917d417bb2c1ed25b7d974b4"
format_stamp: "Formatted at 2026-08-11 15:12:30 on dist-test-slave-1vmg"
server_key: "21d7c3d0b532f4c572dbd39a292f1f70"
server_key_iv: "9335fa93d679810b3dfffb571f90c535"
server_key_version: "encryptionkey@0"
I20260811 15:12:30.450112 26971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:30.467047 26971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:30.470126 26971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:30.470330 26971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:30.470991 26971 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:30.471876 26971 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:12:30.471928 26971 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:30.471988 26971 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:12:30.472015 26971 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:30.480804 26971 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:35691
I20260811 15:12:30.480840 27191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:35691 every 8 connection(s)
I20260811 15:12:30.481745 26971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:12:30.482832 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 26971
I20260811 15:12:30.482995 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:12:30.483314 30752 external_mini_cluster.cc:1428] Setting key 0bfde9fa9f18deef58f1f9b00305355a
Aug 11 15:12:30 dist-test-slave-1vmg krb5kdc[26340](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461150, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:12:30.494517 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:30.483659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:59997 (local address 127.30.8.62:40381)
0811 15:12:30.483826 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:30.483831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:30.485073 (+ 1242us) server_negotiation.cc:407] Connection header received
0811 15:12:30.486000 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:30.486002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:30.486064 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:30.486129 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:30.487292 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.487821 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.488881 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.489097 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.489164 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:30.491640 (+ 2476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:30.491643 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:30.491645 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:30.491667 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:30.492795 (+ 1128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:30.493235 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:30.493237 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:30.493238 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:30.493305 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:30.493565 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:30.493567 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:30.493568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:30.493736 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:30.493804 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:30.494281 (+ 477us) server_negotiation.cc:299] Negotiation successful
0811 15:12:30.494387 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 15:12:30.495364 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:30.484123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:40381 (local address 127.30.8.3:59997)
0811 15:12:30.484898 (+ 775us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:30.484957 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:30.485820 (+ 863us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:30.486263 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:30.486269 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:30.486623 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:30.487177 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:30.487191 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.487911 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.487913 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:30.488796 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:30.488805 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.489224 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.489228 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:30.489421 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:30.489990 (+ 569us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:12:30.490016 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:30.491478 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:30.492890 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:30.492895 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:30.492907 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:30.493162 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:30.493366 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:30.493368 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:30.493369 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:30.493502 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:30.493803 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:30.493808 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:30.494070 (+ 262us) client_negotiation.cc:769] Sending connection context
0811 15:12:30.494229 (+ 159us) client_negotiation.cc:240] Negotiation successful
0811 15:12:30.494490 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"thread_start_us":150,"threads_started":1}
I20260811 15:12:30.496508 27192 heartbeater.cc:347] Connected to a master server at 127.30.8.62:40381
I20260811 15:12:30.496748 27192 heartbeater.cc:464] Registering TS with master...
I20260811 15:12:30.497284 27192 heartbeater.cc:511] Master 127.30.8.62:40381 requested a full tablet report, sending...
I20260811 15:12:30.498281 26418 ts_manager.cc:194] Registered new tserver with Master: 7bd1b569917d417bb2c1ed25b7d974b4 (127.30.8.3:35691)
I20260811 15:12:30.498847 26418 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:59997
I20260811 15:12:30.508881 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 15:12:30.517944 26483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:30.510959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46556 (local address 127.30.8.62:40381)
0811 15:12:30.511108 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:30.511113 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:30.511231 (+ 118us) server_negotiation.cc:407] Connection header received
0811 15:12:30.511365 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:30.511369 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:30.511417 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:30.511484 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:30.512211 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.512792 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.513619 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.513835 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.513924 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:30.515252 (+ 1328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:30.515256 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:30.515260 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:30.515285 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:30.516362 (+ 1077us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:30.516751 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:30.516755 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:30.516757 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:30.516835 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:30.517080 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:30.517083 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:30.517085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:30.517313 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:30.517436 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:30.517671 (+ 235us) server_negotiation.cc:299] Negotiation successful
0811 15:12:30.517786 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 15:12:30.520999 26418 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46556:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:12:30.524161 26418 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:12:30.538550 27213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:30.533182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56692 (local address 127.30.8.1:39053)
0811 15:12:30.534022 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:30.534037 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:30.534122 (+ 85us) server_negotiation.cc:407] Connection header received
0811 15:12:30.534206 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:30.534210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:30.534352 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:30.534820 (+ 468us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:30.534997 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.535999 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.537322 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.537879 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.537976 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:30.538133 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:30.538296 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 15:12:30.538387 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":739,"thread_start_us":91,"threads_started":1}
I20260811 15:12:30.538546 27210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:30.532685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:39053 (local address 127.0.0.1:56692)
0811 15:12:30.533440 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:30.533453 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:30.533544 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:30.534530 (+ 986us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:30.534533 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:30.534554 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:30.534655 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:30.534675 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.536128 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.536131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:30.537211 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:30.537222 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.537991 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.537994 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:30.538109 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:30.538150 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 15:12:30.538257 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 15:12:30.538386 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":692,"mutex_wait_us":68,"thread_start_us":133,"threads_started":1}
I20260811 15:12:30.538892 27212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:30.532428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:35691 (local address 127.0.0.1:50374)
0811 15:12:30.533121 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:30.533173 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:30.533318 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:30.534056 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:30.534064 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:30.534091 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:30.534231 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:30.534237 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.535671 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.535674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:30.537078 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:30.537091 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.538345 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.538349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:30.538544 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:30.538588 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 15:12:30.538653 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 15:12:30.538748 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"mutex_wait_us":140,"thread_start_us":88,"threads_started":1}
I20260811 15:12:30.539126 27215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:30.533603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51056 (local address 127.30.8.2:40713)
0811 15:12:30.534053 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:30.534066 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:30.534087 (+ 21us) server_negotiation.cc:407] Connection header received
0811 15:12:30.534155 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:30.534160 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:30.534332 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:30.534468 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:30.534668 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.535430 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.537230 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.538036 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.538142 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:30.538278 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:30.538829 (+ 551us) server_negotiation.cc:299] Negotiation successful
0811 15:12:30.538973 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":120,"threads_started":1}
I20260811 15:12:30.539209 27214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:30.533197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:40713 (local address 127.0.0.1:51056)
0811 15:12:30.533723 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:30.533734 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:30.533804 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:30.534462 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:30.534465 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:30.534481 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:30.534578 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:30.534585 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.535558 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.535561 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:30.537092 (+ 1531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:30.537099 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.538345 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.538349 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:30.538548 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:30.538583 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 15:12:30.538653 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 15:12:30.538748 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":114,"thread_start_us":86,"threads_started":1}
I20260811 15:12:30.539592 27211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:30.532599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50374 (local address 127.30.8.3:35691)
0811 15:12:30.533069 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:30.533081 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:30.533462 (+ 381us) server_negotiation.cc:407] Connection header received
0811 15:12:30.533548 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:30.533553 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:30.533769 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:30.533914 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:30.534347 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.535538 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.537230 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.538045 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.538142 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:30.538278 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:30.539378 (+ 1100us) server_negotiation.cc:299] Negotiation successful
0811 15:12:30.539467 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":176,"threads_started":1}
I20260811 15:12:30.541975 26630 tablet_service.cc:1467] Processing CreateTablet for tablet a35b68fbd2b54abab39fd3bb35442cb2 (DEFAULT_TABLE table=test-table [id=08b1a8f04e0f42828b676014f7fb0b18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:12:30.542184 26854 tablet_service.cc:1467] Processing CreateTablet for tablet a35b68fbd2b54abab39fd3bb35442cb2 (DEFAULT_TABLE table=test-table [id=08b1a8f04e0f42828b676014f7fb0b18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:12:30.542184 27083 tablet_service.cc:1467] Processing CreateTablet for tablet a35b68fbd2b54abab39fd3bb35442cb2 (DEFAULT_TABLE table=test-table [id=08b1a8f04e0f42828b676014f7fb0b18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:12:30.543126 26630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a35b68fbd2b54abab39fd3bb35442cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:30.543231 26854 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a35b68fbd2b54abab39fd3bb35442cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:30.543231 27083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a35b68fbd2b54abab39fd3bb35442cb2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:30.550102 27216 tablet_bootstrap.cc:492] T a35b68fbd2b54abab39fd3bb35442cb2 P 6059fc590be24071a353ec55ae1b0665: Bootstrap starting.
I20260811 15:12:30.551038 27217 tablet_bootstrap.cc:492] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4: Bootstrap starting.
I20260811 15:12:30.551271 27218 tablet_bootstrap.cc:492] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8: Bootstrap starting.
I20260811 15:12:30.552699 27217 tablet_bootstrap.cc:654] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:30.552698 27216 tablet_bootstrap.cc:654] T a35b68fbd2b54abab39fd3bb35442cb2 P 6059fc590be24071a353ec55ae1b0665: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:30.553223 27218 tablet_bootstrap.cc:654] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:30.553851 27217 log.cc:824] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:30.553947 27216 log.cc:824] T a35b68fbd2b54abab39fd3bb35442cb2 P 6059fc590be24071a353ec55ae1b0665: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:30.554378 27218 log.cc:824] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:30.556035 27217 tablet_bootstrap.cc:492] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4: No bootstrap required, opened a new log
I20260811 15:12:30.556052 27216 tablet_bootstrap.cc:492] T a35b68fbd2b54abab39fd3bb35442cb2 P 6059fc590be24071a353ec55ae1b0665: No bootstrap required, opened a new log
I20260811 15:12:30.556221 27216 ts_tablet_manager.cc:1397] T a35b68fbd2b54abab39fd3bb35442cb2 P 6059fc590be24071a353ec55ae1b0665: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 15:12:30.556219 27217 ts_tablet_manager.cc:1397] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 15:12:30.559122 27216 raft_consensus.cc:348] T a35b68fbd2b54abab39fd3bb35442cb2 P 6059fc590be24071a353ec55ae1b0665 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd1b569917d417bb2c1ed25b7d974b4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35691 } } peers { permanent_uuid: "0372f82447b14b9babbf379e01f1ccd8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40713 } } peers { permanent_uuid: "6059fc590be24071a353ec55ae1b0665" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 39053 } }
I20260811 15:12:30.559139 27217 raft_consensus.cc:348] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd1b569917d417bb2c1ed25b7d974b4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35691 } } peers { permanent_uuid: "0372f82447b14b9babbf379e01f1ccd8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40713 } } peers { permanent_uuid: "6059fc590be24071a353ec55ae1b0665" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 39053 } }
I20260811 15:12:30.559360 27216 raft_consensus.cc:374] T a35b68fbd2b54abab39fd3bb35442cb2 P 6059fc590be24071a353ec55ae1b0665 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:30.559401 27217 raft_consensus.cc:374] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:30.559435 27216 raft_consensus.cc:729] T a35b68fbd2b54abab39fd3bb35442cb2 P 6059fc590be24071a353ec55ae1b0665 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6059fc590be24071a353ec55ae1b0665, State: Initialized, Role: FOLLOWER
I20260811 15:12:30.559485 27217 raft_consensus.cc:729] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bd1b569917d417bb2c1ed25b7d974b4, State: Initialized, Role: FOLLOWER
I20260811 15:12:30.559954 27217 consensus_queue.cc:260] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd1b569917d417bb2c1ed25b7d974b4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35691 } } peers { permanent_uuid: "0372f82447b14b9babbf379e01f1ccd8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40713 } } peers { permanent_uuid: "6059fc590be24071a353ec55ae1b0665" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 39053 } }
I20260811 15:12:30.559966 27216 consensus_queue.cc:260] T a35b68fbd2b54abab39fd3bb35442cb2 P 6059fc590be24071a353ec55ae1b0665 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd1b569917d417bb2c1ed25b7d974b4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35691 } } peers { permanent_uuid: "0372f82447b14b9babbf379e01f1ccd8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40713 } } peers { permanent_uuid: "6059fc590be24071a353ec55ae1b0665" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 39053 } }
I20260811 15:12:30.560607 27192 heartbeater.cc:503] Master 127.30.8.62:40381 was elected leader, sending a full tablet report...
I20260811 15:12:30.560940 27218 tablet_bootstrap.cc:492] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8: No bootstrap required, opened a new log
I20260811 15:12:30.560959 27216 ts_tablet_manager.cc:1428] T a35b68fbd2b54abab39fd3bb35442cb2 P 6059fc590be24071a353ec55ae1b0665: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 15:12:30.560958 27217 ts_tablet_manager.cc:1428] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260811 15:12:30.561154 27218 ts_tablet_manager.cc:1397] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260811 15:12:30.561270 26743 heartbeater.cc:503] Master 127.30.8.62:40381 was elected leader, sending a full tablet report...
I20260811 15:12:30.565711 27218 raft_consensus.cc:348] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd1b569917d417bb2c1ed25b7d974b4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35691 } } peers { permanent_uuid: "0372f82447b14b9babbf379e01f1ccd8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40713 } } peers { permanent_uuid: "6059fc590be24071a353ec55ae1b0665" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 39053 } }
I20260811 15:12:30.565974 27218 raft_consensus.cc:374] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:30.566044 27218 raft_consensus.cc:729] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0372f82447b14b9babbf379e01f1ccd8, State: Initialized, Role: FOLLOWER
I20260811 15:12:30.566464 27218 consensus_queue.cc:260] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd1b569917d417bb2c1ed25b7d974b4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35691 } } peers { permanent_uuid: "0372f82447b14b9babbf379e01f1ccd8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40713 } } peers { permanent_uuid: "6059fc590be24071a353ec55ae1b0665" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 39053 } }
I20260811 15:12:30.567088 26962 heartbeater.cc:503] Master 127.30.8.62:40381 was elected leader, sending a full tablet report...
I20260811 15:12:30.567396 27218 ts_tablet_manager.cc:1428] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
W20260811 15:12:30.732695 27193 tablet.cc:2369] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 15:12:30.763096 26965 tablet.cc:2369] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:12:30.764194 27223 raft_consensus.cc:482] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:12:30.764385 27223 raft_consensus.cc:504] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd1b569917d417bb2c1ed25b7d974b4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35691 } } peers { permanent_uuid: "0372f82447b14b9babbf379e01f1ccd8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40713 } } peers { permanent_uuid: "6059fc590be24071a353ec55ae1b0665" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 39053 } }
I20260811 15:12:30.765547 27223 leader_election.cc:288] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0372f82447b14b9babbf379e01f1ccd8 (127.30.8.2:40713), 6059fc590be24071a353ec55ae1b0665 (127.30.8.1:39053)
I20260811 15:12:30.769965 27213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:30.766074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:57241 (local address 127.30.8.1:39053)
0811 15:12:30.766217 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:30.766222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:30.766247 (+ 25us) server_negotiation.cc:407] Connection header received
0811 15:12:30.766311 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:30.766314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:30.766353 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:30.766424 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:30.766754 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.767476 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.768821 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.769431 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.769521 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:30.769578 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:30.769770 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 15:12:30.769828 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 15:12:30.769991 27195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:30.765850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:39053 (local address 127.30.8.3:57241)
0811 15:12:30.765991 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:30.766010 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:30.766144 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:30.766456 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:30.766459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:30.766481 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:30.766603 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:30.766611 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.767583 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.767586 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:30.768677 (+ 1091us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:30.768687 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.769534 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.769538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:30.769647 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:30.769674 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 15:12:30.769753 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 15:12:30.769823 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51,"mutex_wait_us":17}
I20260811 15:12:30.770534 27234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:30.765854 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:40713 (local address 127.30.8.3:47371)
0811 15:12:30.766710 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:30.766723 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:30.766814 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:30.767095 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:30.767098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:30.767119 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:30.767258 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:30.767264 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.768177 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.768181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:30.769208 (+ 1027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:30.769221 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.770161 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.770165 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:30.770258 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:30.770281 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 15:12:30.770347 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 15:12:30.770406 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"mutex_wait_us":362,"thread_start_us":149,"threads_started":1}
I20260811 15:12:30.770553 27215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:30.765958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:47371 (local address 127.30.8.2:40713)
0811 15:12:30.766096 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:30.766101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:30.766799 (+ 698us) server_negotiation.cc:407] Connection header received
0811 15:12:30.766895 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:30.766898 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:30.766970 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:30.767031 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:30.767384 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.768040 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.769323 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.769930 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.770023 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:30.770083 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:30.770378 (+ 295us) server_negotiation.cc:299] Negotiation successful
0811 15:12:30.770450 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 15:12:30.770699 26668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a35b68fbd2b54abab39fd3bb35442cb2" candidate_uuid: "7bd1b569917d417bb2c1ed25b7d974b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6059fc590be24071a353ec55ae1b0665" is_pre_election: true
I20260811 15:12:30.771091 26668 raft_consensus.cc:2454] T a35b68fbd2b54abab39fd3bb35442cb2 P 6059fc590be24071a353ec55ae1b0665 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bd1b569917d417bb2c1ed25b7d974b4 in term 0.
I20260811 15:12:30.771368 26882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a35b68fbd2b54abab39fd3bb35442cb2" candidate_uuid: "7bd1b569917d417bb2c1ed25b7d974b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0372f82447b14b9babbf379e01f1ccd8" is_pre_election: true
I20260811 15:12:30.771628 26882 raft_consensus.cc:2454] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7bd1b569917d417bb2c1ed25b7d974b4 in term 0.
I20260811 15:12:30.771631 27007 leader_election.cc:302] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6059fc590be24071a353ec55ae1b0665, 7bd1b569917d417bb2c1ed25b7d974b4; no voters:
I20260811 15:12:30.771907 27223 raft_consensus.cc:2781] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:12:30.772001 27223 raft_consensus.cc:482] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:12:30.772050 27223 raft_consensus.cc:3037] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:30.773149 27223 raft_consensus.cc:504] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd1b569917d417bb2c1ed25b7d974b4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35691 } } peers { permanent_uuid: "0372f82447b14b9babbf379e01f1ccd8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40713 } } peers { permanent_uuid: "6059fc590be24071a353ec55ae1b0665" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 39053 } }
I20260811 15:12:30.773516 27223 leader_election.cc:288] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [CANDIDATE]: Term 1 election: Requested vote from peers 0372f82447b14b9babbf379e01f1ccd8 (127.30.8.2:40713), 6059fc590be24071a353ec55ae1b0665 (127.30.8.1:39053)
I20260811 15:12:30.773893 26882 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a35b68fbd2b54abab39fd3bb35442cb2" candidate_uuid: "7bd1b569917d417bb2c1ed25b7d974b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0372f82447b14b9babbf379e01f1ccd8"
I20260811 15:12:30.773927 26668 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a35b68fbd2b54abab39fd3bb35442cb2" candidate_uuid: "7bd1b569917d417bb2c1ed25b7d974b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6059fc590be24071a353ec55ae1b0665"
I20260811 15:12:30.774017 26882 raft_consensus.cc:3037] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:30.774035 26668 raft_consensus.cc:3037] T a35b68fbd2b54abab39fd3bb35442cb2 P 6059fc590be24071a353ec55ae1b0665 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:30.775120 26668 raft_consensus.cc:2454] T a35b68fbd2b54abab39fd3bb35442cb2 P 6059fc590be24071a353ec55ae1b0665 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bd1b569917d417bb2c1ed25b7d974b4 in term 1.
I20260811 15:12:30.775156 26882 raft_consensus.cc:2454] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7bd1b569917d417bb2c1ed25b7d974b4 in term 1.
I20260811 15:12:30.775422 27007 leader_election.cc:302] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6059fc590be24071a353ec55ae1b0665, 7bd1b569917d417bb2c1ed25b7d974b4; no voters:
I20260811 15:12:30.775599 27223 raft_consensus.cc:2781] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:30.775833 27223 raft_consensus.cc:686] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [term 1 LEADER]: Becoming Leader. State: Replica: 7bd1b569917d417bb2c1ed25b7d974b4, State: Running, Role: LEADER
I20260811 15:12:30.776180 27223 consensus_queue.cc:237] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd1b569917d417bb2c1ed25b7d974b4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35691 } } peers { permanent_uuid: "0372f82447b14b9babbf379e01f1ccd8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40713 } } peers { permanent_uuid: "6059fc590be24071a353ec55ae1b0665" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 39053 } }
I20260811 15:12:30.779426 26418 catalog_manager.cc:5495] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7bd1b569917d417bb2c1ed25b7d974b4 (127.30.8.3). New cstate: current_term: 1 leader_uuid: "7bd1b569917d417bb2c1ed25b7d974b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7bd1b569917d417bb2c1ed25b7d974b4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 35691 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0372f82447b14b9babbf379e01f1ccd8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40713 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6059fc590be24071a353ec55ae1b0665" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 39053 } health_report { overall_health: UNKNOWN } } }
W20260811 15:12:30.798341 26745 tablet.cc:2369] T a35b68fbd2b54abab39fd3bb35442cb2 P 6059fc590be24071a353ec55ae1b0665: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:12:30.831851 27211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:30.828657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50378 (local address 127.30.8.3:35691)
0811 15:12:30.828789 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:30.828793 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:30.828806 (+ 13us) server_negotiation.cc:407] Connection header received
0811 15:12:30.828884 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:30.828886 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:30.828945 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:30.829032 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:12:30.829224 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.829789 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.830553 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:30.830780 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:30.830849 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:30.831102 (+ 253us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:12:30.831542 (+ 440us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:12:30.831641 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:30.831687 (+ 46us) server_negotiation.cc:299] Negotiation successful
0811 15:12:30.831738 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 15:12:30.837895 26882 raft_consensus.cc:1261] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8 [term 1 FOLLOWER]: Refusing update from remote peer 7bd1b569917d417bb2c1ed25b7d974b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:12:30.837895 26668 raft_consensus.cc:1261] T a35b68fbd2b54abab39fd3bb35442cb2 P 6059fc590be24071a353ec55ae1b0665 [term 1 FOLLOWER]: Refusing update from remote peer 7bd1b569917d417bb2c1ed25b7d974b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:12:30.838568 27237 consensus_queue.cc:1035] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0372f82447b14b9babbf379e01f1ccd8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40713 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:12:30.838762 27223 consensus_queue.cc:1035] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6059fc590be24071a353ec55ae1b0665" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 39053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:12:30.846371 27243 mvcc.cc:205] Tried to move back new op lower bound from 7317344873824239616 to 7317344873581826048. Current Snapshot: MvccSnapshot[applied={T|T < 7317344873824239616}]
I20260811 15:12:30.859910 26418 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:12:30.860117 26418 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46556:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:12:30.863021 26418 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 74bccfe3ab314f989dd19610ad57f8a0: Sending DeleteTablet for 3 replicas of tablet a35b68fbd2b54abab39fd3bb35442cb2
I20260811 15:12:30.864087 27083 tablet_service.cc:1514] Processing DeleteTablet for tablet a35b68fbd2b54abab39fd3bb35442cb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:12:30 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:50374
I20260811 15:12:30.864095 26630 tablet_service.cc:1514] Processing DeleteTablet for tablet a35b68fbd2b54abab39fd3bb35442cb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:12:30 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:56692
I20260811 15:12:30.864902 26854 tablet_service.cc:1514] Processing DeleteTablet for tablet a35b68fbd2b54abab39fd3bb35442cb2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:12:30 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:51056
I20260811 15:12:30.865458 27266 tablet_replica.cc:330] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8: stopping tablet replica
I20260811 15:12:30.865820 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 26520
I20260811 15:12:30.865869 27266 raft_consensus.cc:2229] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:12:30.866771 27266 raft_consensus.cc:2258] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:12:30.870463 27266 ts_tablet_manager.cc:1905] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:12:30.873441 27266 ts_tablet_manager.cc:1918] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:12:30.873540 27266 log.cc:1196] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/a35b68fbd2b54abab39fd3bb35442cb2
I20260811 15:12:30.873867 27266 ts_tablet_manager.cc:1939] T a35b68fbd2b54abab39fd3bb35442cb2 P 0372f82447b14b9babbf379e01f1ccd8: Deleting consensus metadata
I20260811 15:12:30.876150 26395 catalog_manager.cc:4841] TS 0372f82447b14b9babbf379e01f1ccd8 (127.30.8.2:40713): tablet a35b68fbd2b54abab39fd3bb35442cb2 (table test-table [id=08b1a8f04e0f42828b676014f7fb0b18]) successfully deleted
I20260811 15:12:30.879081 27262 tablet_replica.cc:330] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4: stopping tablet replica
I20260811 15:12:30.879546 27262 raft_consensus.cc:2229] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:12:30.880246 27262 raft_consensus.cc:2258] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:12:30.881976 27262 ts_tablet_manager.cc:1905] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:12:30.884910 27262 ts_tablet_manager.cc:1918] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:12:30.885079 27262 log.cc:1196] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/a35b68fbd2b54abab39fd3bb35442cb2
I20260811 15:12:30.885462 27262 ts_tablet_manager.cc:1939] T a35b68fbd2b54abab39fd3bb35442cb2 P 7bd1b569917d417bb2c1ed25b7d974b4: Deleting consensus metadata
I20260811 15:12:30.886961 26395 catalog_manager.cc:4841] TS 7bd1b569917d417bb2c1ed25b7d974b4 (127.30.8.3:35691): tablet a35b68fbd2b54abab39fd3bb35442cb2 (table test-table [id=08b1a8f04e0f42828b676014f7fb0b18]) successfully deleted
W20260811 15:12:30.887370 26399 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.8.1:39053 (error 108)
I20260811 15:12:30.887657 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 26751
I20260811 15:12:30.891520 27212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:30.888363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:39053 (local address 127.0.0.1:56704)
0811 15:12:30.891334 (+ 2971us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:30.891420 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.1:39053: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2865}
W20260811 15:12:30.892443 26399 catalog_manager.cc:4584] TS 6059fc590be24071a353ec55ae1b0665 (127.30.8.1:39053): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a35b68fbd2b54abab39fd3bb35442cb2: Network error: Client connection negotiation failed: client connection to 127.30.8.1:39053: connect: Connection refused (error 111)
I20260811 15:12:30.898717 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 26971
I20260811 15:12:30.915340 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 26363
2026-08-11T15:12:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2659 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T15:12:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:12:30Z Disabled control of system clock
I20260811 15:12:30.954077 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:45793
--webserver_interface=127.30.8.62
--webserver_port=0
--builtin_ntp_servers=127.30.8.20:41707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:45793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 15:12:31.068066 27273 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:31.068361 27273 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:31.068397 27273 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:31.072492 27273 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:12:31.072568 27273 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:31.072582 27273 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:31.072598 27273 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:12:31.072614 27273 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:12:31.077468 27273 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:41707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:45793
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:45793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27273
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:31.078625 27273 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:31.080147 27273 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:31.088521 27282 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:31.088533 27286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:31.088591 27273 server_base.cc:1034] running on GCE node
W20260811 15:12:31.088523 27281 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:31.089319 27273 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:31.090469 27273 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:31.091621 27273 hybrid_clock.cc:648] HybridClock initialized: now 1786461151091607 us; error 28 us; skew 500 ppm
I20260811 15:12:31.093523 27273 webserver.cc:460] Webserver started at http://127.30.8.62:43897/ using document root <none> and password file <none>
I20260811 15:12:31.094115 27273 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:31.094180 27273 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:31.094403 27273 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:31.096549 27273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "ad224489b60d41969e074a43ca7e9a4d"
format_stamp: "Formatted at 2026-08-11 15:12:31 on dist-test-slave-1vmg"
server_key: "30b54c94ae5ca8410e84e7805639448d"
server_key_iv: "e634c028b0522a38d1de8139e13b2567"
server_key_version: "encryptionkey@0"
I20260811 15:12:31.097105 27273 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "ad224489b60d41969e074a43ca7e9a4d"
format_stamp: "Formatted at 2026-08-11 15:12:31 on dist-test-slave-1vmg"
server_key: "30b54c94ae5ca8410e84e7805639448d"
server_key_iv: "e634c028b0522a38d1de8139e13b2567"
server_key_version: "encryptionkey@0"
I20260811 15:12:31.100858 27273 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 15:12:31.103256 27292 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:31.104439 27273 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:12:31.104569 27273 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "ad224489b60d41969e074a43ca7e9a4d"
format_stamp: "Formatted at 2026-08-11 15:12:31 on dist-test-slave-1vmg"
server_key: "30b54c94ae5ca8410e84e7805639448d"
server_key_iv: "e634c028b0522a38d1de8139e13b2567"
server_key_version: "encryptionkey@0"
I20260811 15:12:31.104681 27273 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:31.137018 27273 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:31.137786 27273 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:31.137975 27273 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:31.145049 27273 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:45793
I20260811 15:12:31.145062 27372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:45793 every 8 connection(s)
I20260811 15:12:31.146054 27273 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:12:31.149271 27373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:31.150560 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 27273
I20260811 15:12:31.150701 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:12:31.151044 30752 external_mini_cluster.cc:1428] Setting key 1a9f66be8476826b24aecdaa7c136ea7
I20260811 15:12:31.155709 27373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d: Bootstrap starting.
I20260811 15:12:31.157963 27373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:31.158756 27373 log.cc:824] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:31.160346 27376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:31.152546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44564 (local address 127.30.8.62:45793)
0811 15:12:31.153199 (+ 653us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:31.153214 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:31.153250 (+ 36us) server_negotiation.cc:407] Connection header received
0811 15:12:31.153899 (+ 649us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:31.153916 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:31.154302 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:31.154618 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:31.155136 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.156193 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.157042 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.157516 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.157675 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:31.158540 (+ 865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:31.158545 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:31.158560 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:31.159042 (+ 482us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:31.159106 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:31.159128 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:31.159217 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:31.159355 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 15:12:31.159624 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":128,"threads_started":1}
I20260811 15:12:31.160756 27373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d: No bootstrap required, opened a new log
I20260811 15:12:31.164410 27373 raft_consensus.cc:348] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad224489b60d41969e074a43ca7e9a4d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45793 } }
I20260811 15:12:31.164642 27373 raft_consensus.cc:374] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:31.164702 27373 raft_consensus.cc:729] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ad224489b60d41969e074a43ca7e9a4d, State: Initialized, Role: FOLLOWER
I20260811 15:12:31.165148 27373 consensus_queue.cc:260] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad224489b60d41969e074a43ca7e9a4d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45793 } }
I20260811 15:12:31.165307 27373 raft_consensus.cc:388] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:12:31.165385 27373 raft_consensus.cc:482] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:12:31.165490 27373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:31.166432 27373 raft_consensus.cc:504] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad224489b60d41969e074a43ca7e9a4d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45793 } }
I20260811 15:12:31.166759 27373 leader_election.cc:302] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ad224489b60d41969e074a43ca7e9a4d; no voters:
I20260811 15:12:31.167097 27373 leader_election.cc:288] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:12:31.167166 27380 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:31.167387 27380 raft_consensus.cc:686] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d [term 1 LEADER]: Becoming Leader. State: Replica: ad224489b60d41969e074a43ca7e9a4d, State: Running, Role: LEADER
I20260811 15:12:31.167737 27380 consensus_queue.cc:237] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad224489b60d41969e074a43ca7e9a4d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45793 } }
I20260811 15:12:31.168450 27373 sys_catalog.cc:564] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:12:31.169466 27381 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ad224489b60d41969e074a43ca7e9a4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad224489b60d41969e074a43ca7e9a4d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45793 } } }
I20260811 15:12:31.169610 27381 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:31.169967 27403 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:12:31.169934 27387 sys_catalog.cc:455] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d [sys.catalog]: SysCatalogTable state changed. Reason: New leader ad224489b60d41969e074a43ca7e9a4d. Latest consensus state: current_term: 1 leader_uuid: "ad224489b60d41969e074a43ca7e9a4d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ad224489b60d41969e074a43ca7e9a4d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45793 } } }
I20260811 15:12:31.170768 27387 sys_catalog.cc:458] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:31.172816 27403 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:12:31.178241 27403 catalog_manager.cc:1347] Generated new cluster ID: 6f8f5a12b8a64598a09e0217bd566673
I20260811 15:12:31.178324 27403 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:12:31.186547 27403 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:12:31.187534 27403 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:12:31.193045 27403 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d: Generated new TSK 0
I20260811 15:12:31.193760 27403 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:12:31.201756 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45793
--builtin_ntp_servers=127.30.8.20:41707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 15:12:31.317119 27417 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:31.317409 27417 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:31.317466 27417 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:31.321527 27417 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:31.321594 27417 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:31.321672 27417 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:12:31.327174 27417 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:41707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27417
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:31.328711 27417 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:31.330492 27417 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:31.341066 27423 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:31.341054 27428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:31.341053 27426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:31.341342 27417 server_base.cc:1034] running on GCE node
I20260811 15:12:31.341753 27417 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:31.342412 27417 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:31.343573 27417 hybrid_clock.cc:648] HybridClock initialized: now 1786461151343558 us; error 27 us; skew 500 ppm
I20260811 15:12:31.345477 27417 webserver.cc:460] Webserver started at http://127.30.8.1:40961/ using document root <none> and password file <none>
I20260811 15:12:31.346052 27417 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:31.346133 27417 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:31.346350 27417 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:31.348453 27417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "3d1275d4da9147749b51c3921558a357"
format_stamp: "Formatted at 2026-08-11 15:12:31 on dist-test-slave-1vmg"
server_key: "f19d1adf13d4d630cd8f2c7da1239826"
server_key_iv: "ccbfbc1c493dbb7786c32bb44d076116"
server_key_version: "encryptionkey@0"
I20260811 15:12:31.348996 27417 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "3d1275d4da9147749b51c3921558a357"
format_stamp: "Formatted at 2026-08-11 15:12:31 on dist-test-slave-1vmg"
server_key: "f19d1adf13d4d630cd8f2c7da1239826"
server_key_iv: "ccbfbc1c493dbb7786c32bb44d076116"
server_key_version: "encryptionkey@0"
I20260811 15:12:31.352739 27417 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.005s sys 0.000s
I20260811 15:12:31.355165 27445 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:31.356365 27417 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:12:31.356492 27417 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "3d1275d4da9147749b51c3921558a357"
format_stamp: "Formatted at 2026-08-11 15:12:31 on dist-test-slave-1vmg"
server_key: "f19d1adf13d4d630cd8f2c7da1239826"
server_key_iv: "ccbfbc1c493dbb7786c32bb44d076116"
server_key_version: "encryptionkey@0"
I20260811 15:12:31.356599 27417 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:31.366284 27417 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:31.366997 27417 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:31.367178 27417 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:31.367763 27417 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:31.368734 27417 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:12:31.368784 27417 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:31.368839 27417 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:12:31.368860 27417 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:31.377416 27417 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:44787
I20260811 15:12:31.377439 27642 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:44787 every 8 connection(s)
I20260811 15:12:31.378378 27417 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:12:31.379221 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 27417
I20260811 15:12:31.379351 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:12:31.379673 30752 external_mini_cluster.cc:1428] Setting key dbb730f539fefc1ae7a506578b09b20c
I20260811 15:12:31.381781 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45793
--builtin_ntp_servers=127.30.8.20:41707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 15:12:31.388696 27376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:31.380390 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:44479 (local address 127.30.8.62:45793)
0811 15:12:31.380537 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:31.380542 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:31.381844 (+ 1302us) server_negotiation.cc:407] Connection header received
0811 15:12:31.383653 (+ 1809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:31.383657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:31.383720 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:31.383819 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:31.384718 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.385478 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.386607 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.386978 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.387088 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:31.388189 (+ 1101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:31.388193 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:31.388197 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:31.388214 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:31.388238 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:31.388249 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:31.388329 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:31.388448 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 15:12:31.388530 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 15:12:31.389595 27646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:31.380853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45793 (local address 127.30.8.1:44479)
0811 15:12:31.381678 (+ 825us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:31.381735 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:31.383415 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:31.383970 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:31.383977 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:31.384064 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:31.384572 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:31.384586 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.385593 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.385595 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:31.386482 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:31.386491 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.387130 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.387134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:31.387288 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:31.387910 (+ 622us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:12:31.387912 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:31.387955 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:12:31.388026 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:31.388326 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:31.388330 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:31.388334 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 15:12:31.388424 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 15:12:31.388665 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":164,"threads_started":1}
I20260811 15:12:31.390852 27644 heartbeater.cc:347] Connected to a master server at 127.30.8.62:45793
I20260811 15:12:31.391116 27644 heartbeater.cc:464] Registering TS with master...
I20260811 15:12:31.391681 27644 heartbeater.cc:511] Master 127.30.8.62:45793 requested a full tablet report, sending...
I20260811 15:12:31.393200 27315 ts_manager.cc:194] Registered new tserver with Master: 3d1275d4da9147749b51c3921558a357 (127.30.8.1:44787)
I20260811 15:12:31.394351 27315 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.8.1:44479
W20260811 15:12:31.498126 27648 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:31.498420 27648 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:31.498474 27648 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:31.502517 27648 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:31.502600 27648 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:31.502681 27648 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:12:31.507546 27648 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:41707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27648
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:31.508813 27648 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:31.510288 27648 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:31.519963 27661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:31.520015 27648 server_base.cc:1034] running on GCE node
W20260811 15:12:31.520037 27657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:31.519963 27658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:31.520655 27648 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:31.521318 27648 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:31.522473 27648 hybrid_clock.cc:648] HybridClock initialized: now 1786461151522458 us; error 30 us; skew 500 ppm
I20260811 15:12:31.524369 27648 webserver.cc:460] Webserver started at http://127.30.8.2:33161/ using document root <none> and password file <none>
I20260811 15:12:31.524946 27648 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:31.525009 27648 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:31.525233 27648 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:31.527360 27648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "8843aac16e964e2bbd513d42941ff0bd"
format_stamp: "Formatted at 2026-08-11 15:12:31 on dist-test-slave-1vmg"
server_key: "8ae3eea418de803fdbbe313b8371ee0c"
server_key_iv: "f9453cec360c8f762c04fcb54e42453e"
server_key_version: "encryptionkey@0"
I20260811 15:12:31.527894 27648 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "8843aac16e964e2bbd513d42941ff0bd"
format_stamp: "Formatted at 2026-08-11 15:12:31 on dist-test-slave-1vmg"
server_key: "8ae3eea418de803fdbbe313b8371ee0c"
server_key_iv: "f9453cec360c8f762c04fcb54e42453e"
server_key_version: "encryptionkey@0"
I20260811 15:12:31.531654 27648 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 15:12:31.534135 27675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:31.535395 27648 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:12:31.535516 27648 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "8843aac16e964e2bbd513d42941ff0bd"
format_stamp: "Formatted at 2026-08-11 15:12:31 on dist-test-slave-1vmg"
server_key: "8ae3eea418de803fdbbe313b8371ee0c"
server_key_iv: "f9453cec360c8f762c04fcb54e42453e"
server_key_version: "encryptionkey@0"
I20260811 15:12:31.535627 27648 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:31.548863 27648 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:31.549599 27648 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:31.549774 27648 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:31.550381 27648 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:31.551432 27648 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:12:31.551483 27648 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:31.551527 27648 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:12:31.551558 27648 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:31.560305 27648 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:43207
I20260811 15:12:31.560336 27844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:43207 every 8 connection(s)
I20260811 15:12:31.561280 27648 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:12:31.569725 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 27648
I20260811 15:12:31.569866 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:12:31.570218 30752 external_mini_cluster.cc:1428] Setting key a0c9c48e32f4aa15f1941b11a95bc426
I20260811 15:12:31.570387 27376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:31.563136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:48767 (local address 127.30.8.62:45793)
0811 15:12:31.563292 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:31.563297 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:31.564378 (+ 1081us) server_negotiation.cc:407] Connection header received
0811 15:12:31.565700 (+ 1322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:31.565703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:31.565749 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:31.565823 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:31.566696 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.567277 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.568432 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.568650 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.568727 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:31.569874 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:31.569878 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:31.569881 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:31.569897 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:31.569920 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:31.569931 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:31.570004 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:31.570142 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 15:12:31.570235 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 15:12:31.571481 27850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:31.563560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45793 (local address 127.30.8.2:48767)
0811 15:12:31.564224 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:31.564266 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:31.565514 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:31.565977 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:31.565984 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:31.566067 (+ 83us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:31.566583 (+ 516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:31.566596 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.567382 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.567385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:31.568341 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:31.568351 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.568765 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.568768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:31.568914 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:31.569596 (+ 682us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:12:31.569598 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:31.569641 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:12:31.569712 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:31.570005 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:31.570014 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:31.570019 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 15:12:31.570131 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 15:12:31.570427 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":121,"threads_started":1}
I20260811 15:12:31.572355 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45793
--builtin_ntp_servers=127.30.8.20:41707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 15:12:31.572672 27846 heartbeater.cc:347] Connected to a master server at 127.30.8.62:45793
I20260811 15:12:31.572925 27846 heartbeater.cc:464] Registering TS with master...
I20260811 15:12:31.573556 27846 heartbeater.cc:511] Master 127.30.8.62:45793 requested a full tablet report, sending...
I20260811 15:12:31.574674 27315 ts_manager.cc:194] Registered new tserver with Master: 8843aac16e964e2bbd513d42941ff0bd (127.30.8.2:43207)
I20260811 15:12:31.575292 27315 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.8.2:48767
W20260811 15:12:31.689499 27854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:31.689797 27854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:31.689841 27854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:31.693982 27854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:31.694056 27854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:31.694121 27854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:12:31.699137 27854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:41707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:45793
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:31.700335 27854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:31.701779 27854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:31.711566 27871 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:31.711566 27874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:31.711570 27867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:31.711886 27854 server_base.cc:1034] running on GCE node
I20260811 15:12:31.712354 27854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:31.713028 27854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:31.714222 27854 hybrid_clock.cc:648] HybridClock initialized: now 1786461151714205 us; error 33 us; skew 500 ppm
I20260811 15:12:31.716171 27854 webserver.cc:460] Webserver started at http://127.30.8.3:37677/ using document root <none> and password file <none>
I20260811 15:12:31.716761 27854 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:31.716833 27854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:31.717046 27854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:31.719254 27854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "f76462cc50fd45f980c1cccb601cfcc9"
format_stamp: "Formatted at 2026-08-11 15:12:31 on dist-test-slave-1vmg"
server_key: "860eefa1c2451c290665e31d66cb9301"
server_key_iv: "9acd0ad497fbf607b6783ff81c614935"
server_key_version: "encryptionkey@0"
I20260811 15:12:31.719933 27854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "f76462cc50fd45f980c1cccb601cfcc9"
format_stamp: "Formatted at 2026-08-11 15:12:31 on dist-test-slave-1vmg"
server_key: "860eefa1c2451c290665e31d66cb9301"
server_key_iv: "9acd0ad497fbf607b6783ff81c614935"
server_key_version: "encryptionkey@0"
I20260811 15:12:31.723739 27854 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 15:12:31.726243 27883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:31.727505 27854 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:12:31.727626 27854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "f76462cc50fd45f980c1cccb601cfcc9"
format_stamp: "Formatted at 2026-08-11 15:12:31 on dist-test-slave-1vmg"
server_key: "860eefa1c2451c290665e31d66cb9301"
server_key_iv: "9acd0ad497fbf607b6783ff81c614935"
server_key_version: "encryptionkey@0"
I20260811 15:12:31.727732 27854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:31.749946 27854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:31.750708 27854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:31.750895 27854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:31.751565 27854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:31.752650 27854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:12:31.752714 27854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:31.752770 27854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:12:31.752800 27854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:31.761759 27854 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:43529
I20260811 15:12:31.761822 28074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:43529 every 8 connection(s)
I20260811 15:12:31.762722 27854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:12:31.770345 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 27854
I20260811 15:12:31.770498 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:12:31.770843 30752 external_mini_cluster.cc:1428] Setting key ac24c58be86f36032c4fc9374ce1b92b
I20260811 15:12:31.772341 27376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:31.764635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:44641 (local address 127.30.8.62:45793)
0811 15:12:31.764784 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:31.764788 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:31.765836 (+ 1048us) server_negotiation.cc:407] Connection header received
0811 15:12:31.767241 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:31.767244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:31.767292 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:31.767366 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:31.768243 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.768845 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.770080 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.770401 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.770534 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:31.771796 (+ 1262us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:31.771800 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:31.771803 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:31.771817 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:31.771843 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:31.771854 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:31.771929 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:31.772149 (+ 220us) server_negotiation.cc:299] Negotiation successful
0811 15:12:31.772215 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 15:12:31.773375 28077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:31.765017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:45793 (local address 127.30.8.3:44641)
0811 15:12:31.765702 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:31.765743 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:31.767052 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:31.767518 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:31.767524 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:31.767617 (+ 93us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:31.768131 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:31.768144 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.768951 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.768953 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:31.769953 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:31.769964 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.770576 (+ 612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.770581 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:31.770770 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:31.771470 (+ 700us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:12:31.771472 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:31.771530 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:12:31.771631 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:31.771943 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:31.771953 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:31.771961 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 15:12:31.772122 (+ 161us) client_negotiation.cc:240] Negotiation successful
0811 15:12:31.772442 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":124,"threads_started":1}
I20260811 15:12:31.774475 28075 heartbeater.cc:347] Connected to a master server at 127.30.8.62:45793
I20260811 15:12:31.774693 28075 heartbeater.cc:464] Registering TS with master...
I20260811 15:12:31.775242 28075 heartbeater.cc:511] Master 127.30.8.62:45793 requested a full tablet report, sending...
I20260811 15:12:31.776233 27315 ts_manager.cc:194] Registered new tserver with Master: f76462cc50fd45f980c1cccb601cfcc9 (127.30.8.3:43529)
I20260811 15:12:31.776777 27315 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.8.3:44641
I20260811 15:12:31.784958 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 15:12:31.790905 27376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:31.787019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44576 (local address 127.30.8.62:45793)
0811 15:12:31.787166 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:31.787172 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:31.787336 (+ 164us) server_negotiation.cc:407] Connection header received
0811 15:12:31.787473 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:31.787476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:31.787530 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:31.787609 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:31.787906 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.788449 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.789308 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.789542 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.789636 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:31.790430 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:31.790435 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:31.790438 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:31.790453 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:31.790476 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:31.790487 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:31.790551 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:31.790677 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 15:12:31.790742 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 15:12:31.794332 27315 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:44576:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:12:31.797538 27315 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:12:31.812229 28095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:31.805912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:44787 (local address 127.0.0.1:52850)
0811 15:12:31.806787 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:31.806801 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:31.806897 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:31.807876 (+ 979us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:31.807883 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:31.807908 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:31.808147 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:31.808158 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.809629 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.809632 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:31.810794 (+ 1162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:31.810802 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.811619 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.811623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:31.811732 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:31.811772 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 15:12:31.811891 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 15:12:31.812026 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"mutex_wait_us":33,"thread_start_us":104,"threads_started":1}
I20260811 15:12:31.812252 28094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:31.805865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:43529 (local address 127.0.0.1:33496)
0811 15:12:31.806268 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:31.806321 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:31.806454 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:31.808090 (+ 1636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:31.808093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:31.808108 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:31.808225 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:31.808231 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.809323 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.809326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:31.810661 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:31.810673 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.811786 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.811791 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:31.811890 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:31.811921 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 15:12:31.811984 (+ 63us) client_negotiation.cc:240] Negotiation successful
0811 15:12:31.812070 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":21,"thread_start_us":120,"threads_started":1}
I20260811 15:12:31.812302 28098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:31.806886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52850 (local address 127.30.8.1:44787)
0811 15:12:31.807362 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:31.807374 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:31.807403 (+ 29us) server_negotiation.cc:407] Connection header received
0811 15:12:31.807484 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:31.807490 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:31.807702 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:31.807877 (+ 175us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:31.808289 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.809491 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.810898 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.811489 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.811595 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:31.811810 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:31.811929 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 15:12:31.812159 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":138,"threads_started":1}
I20260811 15:12:31.812883 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:31.805989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33496 (local address 127.30.8.3:43529)
0811 15:12:31.807613 (+ 1624us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:31.807626 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:31.807653 (+ 27us) server_negotiation.cc:407] Connection header received
0811 15:12:31.807725 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:31.807728 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:31.807901 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:31.808108 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:31.808344 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.809219 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.810814 (+ 1595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.811625 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.812330 (+ 705us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:31.812528 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:31.812586 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 15:12:31.812709 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1493,"thread_start_us":179,"threads_started":1}
I20260811 15:12:31.812935 28097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:31.806613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:43207 (local address 127.0.0.1:58312)
0811 15:12:31.807121 (+ 508us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:31.807135 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:31.807223 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:31.807957 (+ 734us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:31.807960 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:31.807979 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:31.808356 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:31.808362 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.810074 (+ 1712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.810077 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:31.811069 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:31.811082 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.812381 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.812385 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:31.812643 (+ 258us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:31.812669 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 15:12:31.812743 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 15:12:31.812814 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"mutex_wait_us":138,"thread_start_us":113,"threads_started":1}
I20260811 15:12:31.813401 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:31.807085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58312 (local address 127.30.8.2:43207)
0811 15:12:31.807495 (+ 410us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:31.807506 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:31.807524 (+ 18us) server_negotiation.cc:407] Connection header received
0811 15:12:31.807598 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:31.807602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:31.807737 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:31.808040 (+ 303us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:31.808473 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.809916 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.811226 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.812010 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.812125 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:31.812309 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:31.813119 (+ 810us) server_negotiation.cc:299] Negotiation successful
0811 15:12:31.813227 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":133,"threads_started":1}
I20260811 15:12:31.816092 27746 tablet_service.cc:1467] Processing CreateTablet for tablet 6ff83ded19774992b857a0f6c8bf4312 (DEFAULT_TABLE table=test-table [id=d6b5d30651604830baf04f7d83cba78f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:12:31.816151 27962 tablet_service.cc:1467] Processing CreateTablet for tablet 6ff83ded19774992b857a0f6c8bf4312 (DEFAULT_TABLE table=test-table [id=d6b5d30651604830baf04f7d83cba78f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:12:31.816151 27517 tablet_service.cc:1467] Processing CreateTablet for tablet 6ff83ded19774992b857a0f6c8bf4312 (DEFAULT_TABLE table=test-table [id=d6b5d30651604830baf04f7d83cba78f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:12:31.817129 27746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ff83ded19774992b857a0f6c8bf4312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:31.817231 27962 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ff83ded19774992b857a0f6c8bf4312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:31.817231 27517 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6ff83ded19774992b857a0f6c8bf4312. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:31.823292 28100 tablet_bootstrap.cc:492] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd: Bootstrap starting.
I20260811 15:12:31.825016 28101 tablet_bootstrap.cc:492] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9: Bootstrap starting.
I20260811 15:12:31.825503 28102 tablet_bootstrap.cc:492] T 6ff83ded19774992b857a0f6c8bf4312 P 3d1275d4da9147749b51c3921558a357: Bootstrap starting.
I20260811 15:12:31.825780 28100 tablet_bootstrap.cc:654] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:31.826699 28100 log.cc:824] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:31.827163 28101 tablet_bootstrap.cc:654] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:31.827491 28102 tablet_bootstrap.cc:654] T 6ff83ded19774992b857a0f6c8bf4312 P 3d1275d4da9147749b51c3921558a357: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:31.827911 28101 log.cc:824] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:31.828202 28102 log.cc:824] T 6ff83ded19774992b857a0f6c8bf4312 P 3d1275d4da9147749b51c3921558a357: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:31.829233 28100 tablet_bootstrap.cc:492] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd: No bootstrap required, opened a new log
I20260811 15:12:31.829445 28100 ts_tablet_manager.cc:1397] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 15:12:31.829830 28101 tablet_bootstrap.cc:492] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9: No bootstrap required, opened a new log
I20260811 15:12:31.830016 28101 ts_tablet_manager.cc:1397] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 15:12:31.830062 28102 tablet_bootstrap.cc:492] T 6ff83ded19774992b857a0f6c8bf4312 P 3d1275d4da9147749b51c3921558a357: No bootstrap required, opened a new log
I20260811 15:12:31.830237 28102 ts_tablet_manager.cc:1397] T 6ff83ded19774992b857a0f6c8bf4312 P 3d1275d4da9147749b51c3921558a357: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 15:12:31.833750 28102 raft_consensus.cc:348] T 6ff83ded19774992b857a0f6c8bf4312 P 3d1275d4da9147749b51c3921558a357 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76462cc50fd45f980c1cccb601cfcc9" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43529 } } peers { permanent_uuid: "3d1275d4da9147749b51c3921558a357" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44787 } } peers { permanent_uuid: "8843aac16e964e2bbd513d42941ff0bd" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 43207 } }
I20260811 15:12:31.834002 28102 raft_consensus.cc:374] T 6ff83ded19774992b857a0f6c8bf4312 P 3d1275d4da9147749b51c3921558a357 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:31.833839 28100 raft_consensus.cc:348] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76462cc50fd45f980c1cccb601cfcc9" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43529 } } peers { permanent_uuid: "3d1275d4da9147749b51c3921558a357" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44787 } } peers { permanent_uuid: "8843aac16e964e2bbd513d42941ff0bd" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 43207 } }
I20260811 15:12:31.834067 28102 raft_consensus.cc:729] T 6ff83ded19774992b857a0f6c8bf4312 P 3d1275d4da9147749b51c3921558a357 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3d1275d4da9147749b51c3921558a357, State: Initialized, Role: FOLLOWER
I20260811 15:12:31.834115 28100 raft_consensus.cc:374] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:31.833981 28101 raft_consensus.cc:348] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76462cc50fd45f980c1cccb601cfcc9" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43529 } } peers { permanent_uuid: "3d1275d4da9147749b51c3921558a357" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44787 } } peers { permanent_uuid: "8843aac16e964e2bbd513d42941ff0bd" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 43207 } }
I20260811 15:12:31.834173 28100 raft_consensus.cc:729] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8843aac16e964e2bbd513d42941ff0bd, State: Initialized, Role: FOLLOWER
I20260811 15:12:31.834182 28101 raft_consensus.cc:374] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:31.834230 28101 raft_consensus.cc:729] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f76462cc50fd45f980c1cccb601cfcc9, State: Initialized, Role: FOLLOWER
I20260811 15:12:31.834642 28102 consensus_queue.cc:260] T 6ff83ded19774992b857a0f6c8bf4312 P 3d1275d4da9147749b51c3921558a357 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76462cc50fd45f980c1cccb601cfcc9" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43529 } } peers { permanent_uuid: "3d1275d4da9147749b51c3921558a357" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44787 } } peers { permanent_uuid: "8843aac16e964e2bbd513d42941ff0bd" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 43207 } }
I20260811 15:12:31.834657 28101 consensus_queue.cc:260] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76462cc50fd45f980c1cccb601cfcc9" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43529 } } peers { permanent_uuid: "3d1275d4da9147749b51c3921558a357" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44787 } } peers { permanent_uuid: "8843aac16e964e2bbd513d42941ff0bd" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 43207 } }
I20260811 15:12:31.834657 28100 consensus_queue.cc:260] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76462cc50fd45f980c1cccb601cfcc9" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43529 } } peers { permanent_uuid: "3d1275d4da9147749b51c3921558a357" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44787 } } peers { permanent_uuid: "8843aac16e964e2bbd513d42941ff0bd" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 43207 } }
I20260811 15:12:31.835754 28102 ts_tablet_manager.cc:1428] T 6ff83ded19774992b857a0f6c8bf4312 P 3d1275d4da9147749b51c3921558a357: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 15:12:31.835775 28100 ts_tablet_manager.cc:1428] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260811 15:12:31.835775 28101 ts_tablet_manager.cc:1428] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 15:12:31.836158 27644 heartbeater.cc:503] Master 127.30.8.62:45793 was elected leader, sending a full tablet report...
I20260811 15:12:31.836210 28075 heartbeater.cc:503] Master 127.30.8.62:45793 was elected leader, sending a full tablet report...
I20260811 15:12:31.836542 27846 heartbeater.cc:503] Master 127.30.8.62:45793 was elected leader, sending a full tablet report...
W20260811 15:12:31.879593 27645 tablet.cc:2369] T 6ff83ded19774992b857a0f6c8bf4312 P 3d1275d4da9147749b51c3921558a357: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:12:31.961117 28113 raft_consensus.cc:482] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:12:31.961385 28113 raft_consensus.cc:504] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76462cc50fd45f980c1cccb601cfcc9" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43529 } } peers { permanent_uuid: "3d1275d4da9147749b51c3921558a357" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44787 } } peers { permanent_uuid: "8843aac16e964e2bbd513d42941ff0bd" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 43207 } }
I20260811 15:12:31.962605 28113 leader_election.cc:288] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3d1275d4da9147749b51c3921558a357 (127.30.8.1:44787), 8843aac16e964e2bbd513d42941ff0bd (127.30.8.2:43207)
I20260811 15:12:31.967211 28099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:31.963023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:58793 (local address 127.30.8.2:43207)
0811 15:12:31.963153 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:31.963159 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:31.963457 (+ 298us) server_negotiation.cc:407] Connection header received
0811 15:12:31.963662 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:31.963665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:31.963708 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:31.963768 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:31.964073 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.964724 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.966035 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.966574 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.966664 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:31.966730 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:31.966991 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 15:12:31.967063 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 15:12:31.967239 28077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:31.962863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:44787 (local address 127.30.8.3:58731)
0811 15:12:31.962997 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:31.963012 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:31.963113 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:31.963483 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:31.963487 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:31.963532 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:31.963663 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:31.963671 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.964619 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.964621 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:31.965821 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:31.965832 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.966664 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.966668 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:31.966822 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:31.966863 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 15:12:31.966976 (+ 113us) client_negotiation.cc:240] Negotiation successful
0811 15:12:31.967074 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28,"mutex_wait_us":36,"spinlock_wait_cycles":18048}
I20260811 15:12:31.967275 28098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:31.963099 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:58731 (local address 127.30.8.1:44787)
0811 15:12:31.963235 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:31.963240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:31.963253 (+ 13us) server_negotiation.cc:407] Connection header received
0811 15:12:31.963310 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:31.963313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:31.963363 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:31.963449 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:31.963795 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.964506 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.965946 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.966538 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.966665 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:31.966730 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:31.966991 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 15:12:31.967063 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 15:12:31.967247 28115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:31.962903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:43207 (local address 127.30.8.3:58793)
0811 15:12:31.963378 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:31.963391 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:31.963545 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:31.963820 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:31.963823 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:31.963840 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:31.963959 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:31.963965 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.964820 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.964824 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:31.965929 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:31.965937 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:31.966675 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:31.966678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:31.966827 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:31.966874 (+ 47us) client_negotiation.cc:769] Sending connection context
0811 15:12:31.966976 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 15:12:31.967074 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":401,"mutex_wait_us":75,"spinlock_wait_cycles":1792,"thread_start_us":130,"threads_started":1}
I20260811 15:12:31.968040 27780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ff83ded19774992b857a0f6c8bf4312" candidate_uuid: "f76462cc50fd45f980c1cccb601cfcc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8843aac16e964e2bbd513d42941ff0bd" is_pre_election: true
I20260811 15:12:31.968034 27555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ff83ded19774992b857a0f6c8bf4312" candidate_uuid: "f76462cc50fd45f980c1cccb601cfcc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d1275d4da9147749b51c3921558a357" is_pre_election: true
I20260811 15:12:31.968387 27555 raft_consensus.cc:2454] T 6ff83ded19774992b857a0f6c8bf4312 P 3d1275d4da9147749b51c3921558a357 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f76462cc50fd45f980c1cccb601cfcc9 in term 0.
I20260811 15:12:31.968387 27780 raft_consensus.cc:2454] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f76462cc50fd45f980c1cccb601cfcc9 in term 0.
I20260811 15:12:31.968896 27887 leader_election.cc:302] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8843aac16e964e2bbd513d42941ff0bd, f76462cc50fd45f980c1cccb601cfcc9; no voters:
I20260811 15:12:31.969249 28113 raft_consensus.cc:2781] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:12:31.969342 28113 raft_consensus.cc:482] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:12:31.969396 28113 raft_consensus.cc:3037] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:31.970525 28113 raft_consensus.cc:504] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76462cc50fd45f980c1cccb601cfcc9" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43529 } } peers { permanent_uuid: "3d1275d4da9147749b51c3921558a357" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44787 } } peers { permanent_uuid: "8843aac16e964e2bbd513d42941ff0bd" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 43207 } }
I20260811 15:12:31.970875 28113 leader_election.cc:288] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [CANDIDATE]: Term 1 election: Requested vote from peers 3d1275d4da9147749b51c3921558a357 (127.30.8.1:44787), 8843aac16e964e2bbd513d42941ff0bd (127.30.8.2:43207)
I20260811 15:12:31.971231 27780 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ff83ded19774992b857a0f6c8bf4312" candidate_uuid: "f76462cc50fd45f980c1cccb601cfcc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8843aac16e964e2bbd513d42941ff0bd"
I20260811 15:12:31.971361 27555 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6ff83ded19774992b857a0f6c8bf4312" candidate_uuid: "f76462cc50fd45f980c1cccb601cfcc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3d1275d4da9147749b51c3921558a357"
I20260811 15:12:31.971361 27780 raft_consensus.cc:3037] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:31.971485 27555 raft_consensus.cc:3037] T 6ff83ded19774992b857a0f6c8bf4312 P 3d1275d4da9147749b51c3921558a357 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:31.972666 27555 raft_consensus.cc:2454] T 6ff83ded19774992b857a0f6c8bf4312 P 3d1275d4da9147749b51c3921558a357 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f76462cc50fd45f980c1cccb601cfcc9 in term 1.
I20260811 15:12:31.972671 27780 raft_consensus.cc:2454] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f76462cc50fd45f980c1cccb601cfcc9 in term 1.
I20260811 15:12:31.972963 27887 leader_election.cc:302] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8843aac16e964e2bbd513d42941ff0bd, f76462cc50fd45f980c1cccb601cfcc9; no voters:
I20260811 15:12:31.973135 28113 raft_consensus.cc:2781] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:31.973392 28113 raft_consensus.cc:686] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [term 1 LEADER]: Becoming Leader. State: Replica: f76462cc50fd45f980c1cccb601cfcc9, State: Running, Role: LEADER
I20260811 15:12:31.973768 28113 consensus_queue.cc:237] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76462cc50fd45f980c1cccb601cfcc9" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43529 } } peers { permanent_uuid: "3d1275d4da9147749b51c3921558a357" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44787 } } peers { permanent_uuid: "8843aac16e964e2bbd513d42941ff0bd" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 43207 } }
I20260811 15:12:31.977233 27315 catalog_manager.cc:5495] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 reported cstate change: term changed from 0 to 1, leader changed from <none> to f76462cc50fd45f980c1cccb601cfcc9 (127.30.8.3). New cstate: current_term: 1 leader_uuid: "f76462cc50fd45f980c1cccb601cfcc9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f76462cc50fd45f980c1cccb601cfcc9" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 43529 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3d1275d4da9147749b51c3921558a357" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8843aac16e964e2bbd513d42941ff0bd" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 43207 } health_report { overall_health: UNKNOWN } } }
I20260811 15:12:32.008904 28096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:32.005732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33506 (local address 127.30.8.3:43529)
0811 15:12:32.005865 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:32.005869 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:32.005882 (+ 13us) server_negotiation.cc:407] Connection header received
0811 15:12:32.005959 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:32.005962 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:32.006014 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:32.006104 (+ 90us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:12:32.006290 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:32.006786 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:32.007571 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:32.007805 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:32.007888 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:32.008127 (+ 239us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:12:32.008568 (+ 441us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:12:32.008663 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:32.008720 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 15:12:32.008773 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 15:12:32.013695 28076 tablet.cc:2369] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:12:32.015133 27780 raft_consensus.cc:1261] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd [term 1 FOLLOWER]: Refusing update from remote peer f76462cc50fd45f980c1cccb601cfcc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:12:32.015133 27555 raft_consensus.cc:1261] T 6ff83ded19774992b857a0f6c8bf4312 P 3d1275d4da9147749b51c3921558a357 [term 1 FOLLOWER]: Refusing update from remote peer f76462cc50fd45f980c1cccb601cfcc9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:12:32.015856 28118 consensus_queue.cc:1035] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3d1275d4da9147749b51c3921558a357" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:12:32.016043 28113 consensus_queue.cc:1035] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8843aac16e964e2bbd513d42941ff0bd" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 43207 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:12:32.026279 28126 mvcc.cc:205] Tried to move back new op lower bound from 7317344878645768192 to 7317344878487326720. Current Snapshot: MvccSnapshot[applied={T|T < 7317344878645768192}]
I20260811 15:12:32.034089 27315 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:44576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:12:32.034276 27315 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:44576:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:12:32.036712 27315 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ad224489b60d41969e074a43ca7e9a4d: Sending DeleteTablet for 3 replicas of tablet 6ff83ded19774992b857a0f6c8bf4312
I20260811 15:12:32.037483 27962 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ff83ded19774992b857a0f6c8bf4312 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:12:32 UTC) from {username='slave'} at 127.0.0.1:33496
I20260811 15:12:32.037707 27746 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ff83ded19774992b857a0f6c8bf4312 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:12:32 UTC) from {username='slave'} at 127.0.0.1:58312
I20260811 15:12:32.038192 28143 tablet_replica.cc:330] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd: stopping tablet replica
I20260811 15:12:32.038237 27517 tablet_service.cc:1514] Processing DeleteTablet for tablet 6ff83ded19774992b857a0f6c8bf4312 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:12:32 UTC) from {username='slave'} at 127.0.0.1:52850
I20260811 15:12:32.038515 28143 raft_consensus.cc:2229] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:12:32.038779 28144 tablet_replica.cc:330] T 6ff83ded19774992b857a0f6c8bf4312 P 3d1275d4da9147749b51c3921558a357: stopping tablet replica
I20260811 15:12:32.038954 28143 raft_consensus.cc:2258] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:12:32.039251 28142 tablet_replica.cc:330] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9: stopping tablet replica
I20260811 15:12:32.039536 28142 raft_consensus.cc:2229] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:12:32.039753 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 27417
I20260811 15:12:32.039755 28144 raft_consensus.cc:2229] T 6ff83ded19774992b857a0f6c8bf4312 P 3d1275d4da9147749b51c3921558a357 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:12:32.039964 28142 raft_consensus.cc:2258] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:12:32.041842 28142 ts_tablet_manager.cc:1905] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:12:32.044721 28142 ts_tablet_manager.cc:1918] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:12:32.044821 28142 log.cc:1196] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/6ff83ded19774992b857a0f6c8bf4312
I20260811 15:12:32.045113 28142 ts_tablet_manager.cc:1939] T 6ff83ded19774992b857a0f6c8bf4312 P f76462cc50fd45f980c1cccb601cfcc9: Deleting consensus metadata
I20260811 15:12:32.046445 27294 catalog_manager.cc:4841] TS f76462cc50fd45f980c1cccb601cfcc9 (127.30.8.3:43529): tablet 6ff83ded19774992b857a0f6c8bf4312 (table test-table [id=d6b5d30651604830baf04f7d83cba78f]) successfully deleted
I20260811 15:12:32.047153 28143 ts_tablet_manager.cc:1905] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:12:32.049794 28143 ts_tablet_manager.cc:1918] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:12:32.049899 28143 log.cc:1196] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/6ff83ded19774992b857a0f6c8bf4312
I20260811 15:12:32.050213 28143 ts_tablet_manager.cc:1939] T 6ff83ded19774992b857a0f6c8bf4312 P 8843aac16e964e2bbd513d42941ff0bd: Deleting consensus metadata
I20260811 15:12:32.051622 27295 catalog_manager.cc:4841] TS 8843aac16e964e2bbd513d42941ff0bd (127.30.8.2:43207): tablet 6ff83ded19774992b857a0f6c8bf4312 (table test-table [id=d6b5d30651604830baf04f7d83cba78f]) successfully deleted
W20260811 15:12:32.053428 27296 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.8.1:44787 (error 108)
I20260811 15:12:32.053750 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 27648
I20260811 15:12:32.063828 28097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:32.063492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:44787 (local address 127.0.0.1:52856)
0811 15:12:32.063651 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:32.063736 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.1:44787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 15:12:32.064141 27296 catalog_manager.cc:4584] TS 3d1275d4da9147749b51c3921558a357 (127.30.8.1:44787): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6ff83ded19774992b857a0f6c8bf4312: Network error: Client connection negotiation failed: client connection to 127.30.8.1:44787: connect: Connection refused (error 111)
I20260811 15:12:32.065845 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 27854
I20260811 15:12:32.077246 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 27273
2026-08-11T15:12:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1162 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:12:32 dist-test-slave-1vmg krb5kdc[28150](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:12:32 dist-test-slave-1vmg krb5kdc[28150](info): set up 2 sockets
Aug 11 15:12:32 dist-test-slave-1vmg krb5kdc[28150](info): commencing operation
krb5kdc: starting...
W20260811 15:12:33.256507 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:12:33 dist-test-slave-1vmg krb5kdc[28150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:12:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:12:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:12:33.405782 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:38287
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:33741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:38287
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:33.532573 28169 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:33.532872 28169 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:33.532913 28169 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:33.537091 28169 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:12:33.537164 28169 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:33.537184 28169 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:33.537200 28169 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:12:33.537222 28169 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:12:33.542691 28169 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:33741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:38287
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:38287
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28169
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:33.543857 28169 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:33.544915 28169 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:33.553560 28179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:33.553705 28169 server_base.cc:1034] running on GCE node
W20260811 15:12:33.553555 28182 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:33.553998 28185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:33.554548 28169 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:33.555768 28169 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:33.556954 28169 hybrid_clock.cc:648] HybridClock initialized: now 1786461153556930 us; error 39 us; skew 500 ppm
Aug 11 15:12:33 dist-test-slave-1vmg krb5kdc[28150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461153, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:33.562901 28169 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:12:33.564011 28169 webserver.cc:460] Webserver started at http://127.30.8.62:43049/ using document root <none> and password file <none>
I20260811 15:12:33.564610 28169 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:33.564680 28169 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:33.564920 28169 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:33.567132 28169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "fdb4552d922348bea9d05264160d501f"
format_stamp: "Formatted at 2026-08-11 15:12:33 on dist-test-slave-1vmg"
server_key: "c1580c63dc56d5ce9ea531f48ae99b31"
server_key_iv: "212a0d498f53cb6326aa6a122b2244a6"
server_key_version: "encryptionkey@0"
I20260811 15:12:33.567692 28169 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "fdb4552d922348bea9d05264160d501f"
format_stamp: "Formatted at 2026-08-11 15:12:33 on dist-test-slave-1vmg"
server_key: "c1580c63dc56d5ce9ea531f48ae99b31"
server_key_iv: "212a0d498f53cb6326aa6a122b2244a6"
server_key_version: "encryptionkey@0"
I20260811 15:12:33.571554 28169 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 15:12:33.574034 28192 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:33.575400 28169 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:12:33.575539 28169 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "fdb4552d922348bea9d05264160d501f"
format_stamp: "Formatted at 2026-08-11 15:12:33 on dist-test-slave-1vmg"
server_key: "c1580c63dc56d5ce9ea531f48ae99b31"
server_key_iv: "212a0d498f53cb6326aa6a122b2244a6"
server_key_version: "encryptionkey@0"
I20260811 15:12:33.575649 28169 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:33.589634 28169 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:33.593047 28169 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:33.593253 28169 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:33.600698 28169 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:38287
I20260811 15:12:33.600718 28279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:38287 every 8 connection(s)
I20260811 15:12:33.601763 28169 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:12:33.604832 28280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:33.605197 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 28169
I20260811 15:12:33.605341 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:12:33.605657 30752 external_mini_cluster.cc:1428] Setting key eb722649f67cffe4b48f1bdea0c3b11b
I20260811 15:12:33.612263 28280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f: Bootstrap starting.
Aug 11 15:12:33 dist-test-slave-1vmg krb5kdc[28150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:12:33.615111 28280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:33.616022 28280 log.cc:824] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:33.618345 28280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f: No bootstrap required, opened a new log
I20260811 15:12:33.620509 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:33.607144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43712 (local address 127.30.8.62:38287)
0811 15:12:33.607589 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:33.607606 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:33.607644 (+ 38us) server_negotiation.cc:407] Connection header received
0811 15:12:33.608350 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:33.608368 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:33.608705 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:33.609006 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:33.609922 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:33.610981 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:33.612121 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:33.612650 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:33.612782 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:33.615747 (+ 2965us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:33.615753 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:33.615770 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:33.615803 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:33.617543 (+ 1740us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:33.618039 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:33.618043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:33.618049 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:33.618159 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:33.618452 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:33.618456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:33.618458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:33.618956 (+ 498us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:33.619125 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:33.619418 (+ 293us) server_negotiation.cc:299] Negotiation successful
0811 15:12:33.619724 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":134,"threads_started":1}
I20260811 15:12:33.622498 28280 raft_consensus.cc:348] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb4552d922348bea9d05264160d501f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 38287 } }
I20260811 15:12:33.622785 28280 raft_consensus.cc:374] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:33.622856 28280 raft_consensus.cc:729] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fdb4552d922348bea9d05264160d501f, State: Initialized, Role: FOLLOWER
I20260811 15:12:33.623325 28280 consensus_queue.cc:260] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb4552d922348bea9d05264160d501f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 38287 } }
I20260811 15:12:33.623469 28280 raft_consensus.cc:388] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:12:33.623538 28280 raft_consensus.cc:482] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:12:33.623639 28280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:33.624665 28280 raft_consensus.cc:504] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb4552d922348bea9d05264160d501f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 38287 } }
I20260811 15:12:33.625016 28280 leader_election.cc:302] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fdb4552d922348bea9d05264160d501f; no voters:
I20260811 15:12:33.625337 28280 leader_election.cc:288] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:12:33.625445 28289 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:33.625658 28289 raft_consensus.cc:686] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f [term 1 LEADER]: Becoming Leader. State: Replica: fdb4552d922348bea9d05264160d501f, State: Running, Role: LEADER
I20260811 15:12:33.625967 28289 consensus_queue.cc:237] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb4552d922348bea9d05264160d501f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 38287 } }
I20260811 15:12:33.626587 28280 sys_catalog.cc:564] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:12:33.627753 28292 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fdb4552d922348bea9d05264160d501f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb4552d922348bea9d05264160d501f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 38287 } } }
I20260811 15:12:33.627811 28294 sys_catalog.cc:455] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fdb4552d922348bea9d05264160d501f. Latest consensus state: current_term: 1 leader_uuid: "fdb4552d922348bea9d05264160d501f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fdb4552d922348bea9d05264160d501f" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 38287 } } }
I20260811 15:12:33.627990 28294 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:33.628062 28292 sys_catalog.cc:458] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:33.628410 28304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:12:33.631965 28304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:12:33.637915 28304 catalog_manager.cc:1347] Generated new cluster ID: bc94c7fef21d40e193c13d0de748c7d9
I20260811 15:12:33.637995 28304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:12:33.650767 28304 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:12:33.651824 28304 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:12:33.657323 28304 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f: Generated new TSK 0
I20260811 15:12:33.658214 28304 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
I20260811 15:12:33.692162 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:38287
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:33741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:33.816066 28319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:33.816373 28319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:33.816433 28319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:33.820534 28319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:33.820611 28319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:33.820694 28319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:12:33.826267 28319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:33741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:38287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:33.827553 28319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:33.828670 28319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:33.838735 28325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:33.838735 28327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:33.838817 28319 server_base.cc:1034] running on GCE node
W20260811 15:12:33.838755 28329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:33.839452 28319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:33.840171 28319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:33.841349 28319 hybrid_clock.cc:648] HybridClock initialized: now 1786461153841330 us; error 38 us; skew 500 ppm
Aug 11 15:12:33 dist-test-slave-1vmg krb5kdc[28150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461153, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:33.844269 28319 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:12:33.845430 28319 webserver.cc:460] Webserver started at http://127.30.8.1:42833/ using document root <none> and password file <none>
I20260811 15:12:33.846031 28319 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:33.846103 28319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:33.846334 28319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:33.848510 28319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "edffc0a281b34dd4bf02eb29e65716e3"
format_stamp: "Formatted at 2026-08-11 15:12:33 on dist-test-slave-1vmg"
server_key: "0bcba31dbc46b7609222e773b3ae7725"
server_key_iv: "aa5ed7bc523db4ed1d329918da891ca5"
server_key_version: "encryptionkey@0"
I20260811 15:12:33.849074 28319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "edffc0a281b34dd4bf02eb29e65716e3"
format_stamp: "Formatted at 2026-08-11 15:12:33 on dist-test-slave-1vmg"
server_key: "0bcba31dbc46b7609222e773b3ae7725"
server_key_iv: "aa5ed7bc523db4ed1d329918da891ca5"
server_key_version: "encryptionkey@0"
I20260811 15:12:33.853246 28319 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:12:33.855950 28344 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:33.857229 28319 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:12:33.857363 28319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "edffc0a281b34dd4bf02eb29e65716e3"
format_stamp: "Formatted at 2026-08-11 15:12:33 on dist-test-slave-1vmg"
server_key: "0bcba31dbc46b7609222e773b3ae7725"
server_key_iv: "aa5ed7bc523db4ed1d329918da891ca5"
server_key_version: "encryptionkey@0"
I20260811 15:12:33.857472 28319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:33.876797 28319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:33.880429 28319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:33.880604 28319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:33.881209 28319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:33.882169 28319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:12:33.882227 28319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:33.882269 28319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:12:33.882298 28319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:33.891113 28319 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:40279
I20260811 15:12:33.891144 28524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:40279 every 8 connection(s)
I20260811 15:12:33.892127 28319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:12:33.901988 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 28319
I20260811 15:12:33.902163 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:12:33.902530 30752 external_mini_cluster.cc:1428] Setting key 21e18937966c9d4ab808cd5999845d0f
Aug 11 15:12:33 dist-test-slave-1vmg krb5kdc[28150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461153, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:12:33.908047 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:33.894639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:52565 (local address 127.30.8.62:38287)
0811 15:12:33.894825 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:33.894830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:33.895754 (+ 924us) server_negotiation.cc:407] Connection header received
0811 15:12:33.897084 (+ 1330us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:33.897088 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:33.897160 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:33.897239 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:33.898567 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:33.899185 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:33.900380 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:33.900620 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:33.900722 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:33.904113 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:33.904117 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:33.904120 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:33.904152 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:33.905694 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:33.906258 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:33.906262 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:33.906264 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:33.906362 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:33.906779 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:33.906782 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:33.906783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:33.907019 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:33.907102 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:33.907788 (+ 686us) server_negotiation.cc:299] Negotiation successful
0811 15:12:33.907913 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 15:12:33.909214 28529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:33.894887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:38287 (local address 127.30.8.1:52565)
0811 15:12:33.895534 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:33.895584 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:33.896828 (+ 1244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:33.897447 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:33.897455 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:33.897853 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:33.898399 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:33.898414 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:33.899318 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:33.899322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:33.900252 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:33.900262 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:33.900775 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:33.900779 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:33.900948 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:33.901812 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:12:33.901855 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:33.903897 (+ 2042us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:33.905832 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:33.905839 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:33.905868 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:33.906158 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:33.906451 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:33.906454 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:33.906456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:33.906677 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:33.907116 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:33.907123 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:33.907496 (+ 373us) client_negotiation.cc:769] Sending connection context
0811 15:12:33.907740 (+ 244us) client_negotiation.cc:240] Negotiation successful
0811 15:12:33.908138 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"thread_start_us":153,"threads_started":1}
I20260811 15:12:33.910677 28525 heartbeater.cc:347] Connected to a master server at 127.30.8.62:38287
I20260811 15:12:33.910966 28525 heartbeater.cc:464] Registering TS with master...
I20260811 15:12:33.911536 28525 heartbeater.cc:511] Master 127.30.8.62:38287 requested a full tablet report, sending...
I20260811 15:12:33.913204 28219 ts_manager.cc:194] Registered new tserver with Master: edffc0a281b34dd4bf02eb29e65716e3 (127.30.8.1:40279)
WARNING: no policy specified for kudu/127.30.8.2@KRBTEST.COM; defaulting to no policy
I20260811 15:12:33.914443 28219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:52565
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
I20260811 15:12:33.935788 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:38287
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:33741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:34.057451 28533 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:34.057760 28533 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:34.057823 28533 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:34.062134 28533 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:34.062220 28533 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:34.062302 28533 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:12:34.067912 28533 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:33741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:38287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28533
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:34.069123 28533 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:34.070194 28533 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:34.080296 28545 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:34.080340 28542 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:34.082406 28543 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:34.083595 28533 server_base.cc:1034] running on GCE node
I20260811 15:12:34.084132 28533 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:34.084856 28533 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:34.086037 28533 hybrid_clock.cc:648] HybridClock initialized: now 1786461154086020 us; error 34 us; skew 500 ppm
Aug 11 15:12:34 dist-test-slave-1vmg krb5kdc[28150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461154, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:34.088930 28533 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:12:34.089915 28533 webserver.cc:460] Webserver started at http://127.30.8.2:39049/ using document root <none> and password file <none>
I20260811 15:12:34.090517 28533 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:34.090579 28533 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:34.090791 28533 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:34.092988 28533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "b8f9839cdbdd4d67812c2e70b7b4a197"
format_stamp: "Formatted at 2026-08-11 15:12:34 on dist-test-slave-1vmg"
server_key: "54a96a98847c248818130385835696ed"
server_key_iv: "93c3d4c06f1afd4a18d40cb47f3cf5cd"
server_key_version: "encryptionkey@0"
I20260811 15:12:34.093570 28533 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "b8f9839cdbdd4d67812c2e70b7b4a197"
format_stamp: "Formatted at 2026-08-11 15:12:34 on dist-test-slave-1vmg"
server_key: "54a96a98847c248818130385835696ed"
server_key_iv: "93c3d4c06f1afd4a18d40cb47f3cf5cd"
server_key_version: "encryptionkey@0"
I20260811 15:12:34.097992 28533 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 15:12:34.101377 28555 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:34.102897 28533 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 15:12:34.103118 28533 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "b8f9839cdbdd4d67812c2e70b7b4a197"
format_stamp: "Formatted at 2026-08-11 15:12:34 on dist-test-slave-1vmg"
server_key: "54a96a98847c248818130385835696ed"
server_key_iv: "93c3d4c06f1afd4a18d40cb47f3cf5cd"
server_key_version: "encryptionkey@0"
I20260811 15:12:34.103248 28533 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:34.116230 28533 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:34.119691 28533 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:34.119910 28533 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:34.120549 28533 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:34.121457 28533 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:12:34.121507 28533 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:34.121562 28533 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:12:34.121591 28533 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:34.130957 28533 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:42769
I20260811 15:12:34.131004 28733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:42769 every 8 connection(s)
I20260811 15:12:34.131901 28533 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:12:34.134848 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 28533
I20260811 15:12:34.135017 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:12:34.135344 30752 external_mini_cluster.cc:1428] Setting key 7e8340b2ae560ea2323929afa97cbcc7
Aug 11 15:12:34 dist-test-slave-1vmg krb5kdc[28150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461154, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.8.3@KRBTEST.COM; defaulting to no policy
I20260811 15:12:34.148721 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:34.134273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:45003 (local address 127.30.8.62:38287)
0811 15:12:34.134467 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:34.134471 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:34.135673 (+ 1202us) server_negotiation.cc:407] Connection header received
0811 15:12:34.137278 (+ 1605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:34.137282 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:34.137354 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:34.137438 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:34.139196 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.139762 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.141665 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.141974 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.142093 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:34.144967 (+ 2874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:34.144971 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:34.144975 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:34.145008 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:34.146594 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:34.147208 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:34.147212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:34.147214 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:34.147309 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:34.147599 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:34.147601 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:34.147602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:34.147786 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:34.147873 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:34.148466 (+ 593us) server_negotiation.cc:299] Negotiation successful
0811 15:12:34.148583 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.3@KRBTEST.COM" created.
I20260811 15:12:34.149588 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:34.134606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:38287 (local address 127.30.8.2:45003)
0811 15:12:34.135436 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:34.135521 (+ 85us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:34.137005 (+ 1484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:34.137676 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:34.137684 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:34.138188 (+ 504us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:34.138898 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:34.138949 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.139921 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.139925 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:34.141514 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:34.141524 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.142192 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.142197 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:34.142386 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:34.143100 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:12:34.143119 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:34.144751 (+ 1632us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:34.146718 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:34.146723 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:34.146738 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:34.147080 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:34.147382 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:34.147384 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:34.147386 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:34.147525 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:34.147870 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:34.147876 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:34.148214 (+ 338us) client_negotiation.cc:769] Sending connection context
0811 15:12:34.148390 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 15:12:34.148699 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"thread_start_us":144,"threads_started":1}
I20260811 15:12:34.150774 28734 heartbeater.cc:347] Connected to a master server at 127.30.8.62:38287
I20260811 15:12:34.151058 28734 heartbeater.cc:464] Registering TS with master...
I20260811 15:12:34.151607 28734 heartbeater.cc:511] Master 127.30.8.62:38287 requested a full tablet report, sending...
I20260811 15:12:34.152837 28219 ts_manager.cc:194] Registered new tserver with Master: b8f9839cdbdd4d67812c2e70b7b4a197 (127.30.8.2:42769)
I20260811 15:12:34.153415 28219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:45003
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
I20260811 15:12:34.166313 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:38287
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:33741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:34.286134 28741 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:34.286429 28741 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:34.286474 28741 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:34.290526 28741 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:34.290608 28741 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:34.290673 28741 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:12:34.296350 28741 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:33741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:38287
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28741
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:34.297544 28741 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:34.298584 28741 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:34.308588 28749 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:34.308601 28750 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:34.308794 28741 server_base.cc:1034] running on GCE node
W20260811 15:12:34.308624 28754 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:34.309402 28741 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:34.310108 28741 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:34.311281 28741 hybrid_clock.cc:648] HybridClock initialized: now 1786461154311261 us; error 31 us; skew 500 ppm
Aug 11 15:12:34 dist-test-slave-1vmg krb5kdc[28150](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461154, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:34.314128 28741 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:12:34.315152 28741 webserver.cc:460] Webserver started at http://127.30.8.3:40383/ using document root <none> and password file <none>
I20260811 15:12:34.315745 28741 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:34.315817 28741 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:34.316039 28741 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:34.318243 28741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "357c41bab8db4960b37f4d481f0678d5"
format_stamp: "Formatted at 2026-08-11 15:12:34 on dist-test-slave-1vmg"
server_key: "0367e117fd654e2fbc92735df4ce19a8"
server_key_iv: "f1b7eb18f0e56cb64b87cd54582e93b7"
server_key_version: "encryptionkey@0"
I20260811 15:12:34.318828 28741 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "357c41bab8db4960b37f4d481f0678d5"
format_stamp: "Formatted at 2026-08-11 15:12:34 on dist-test-slave-1vmg"
server_key: "0367e117fd654e2fbc92735df4ce19a8"
server_key_iv: "f1b7eb18f0e56cb64b87cd54582e93b7"
server_key_version: "encryptionkey@0"
I20260811 15:12:34.322952 28741 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 15:12:34.325736 28766 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:34.327116 28741 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:12:34.327239 28741 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "357c41bab8db4960b37f4d481f0678d5"
format_stamp: "Formatted at 2026-08-11 15:12:34 on dist-test-slave-1vmg"
server_key: "0367e117fd654e2fbc92735df4ce19a8"
server_key_iv: "f1b7eb18f0e56cb64b87cd54582e93b7"
server_key_version: "encryptionkey@0"
I20260811 15:12:34.327369 28741 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:34.344266 28741 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:34.347411 28741 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:34.347615 28741 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:34.348250 28741 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:34.349150 28741 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:12:34.349202 28741 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:34.349257 28741 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:12:34.349287 28741 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:34.358616 28741 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:41841
I20260811 15:12:34.358651 28952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:41841 every 8 connection(s)
I20260811 15:12:34.359643 28741 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:12:34.364707 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 28741
I20260811 15:12:34.364862 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:12:34.365175 30752 external_mini_cluster.cc:1428] Setting key 294dcb3dd74f640596b85977dee43382
Aug 11 15:12:34 dist-test-slave-1vmg krb5kdc[28150](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461154, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:12:34.373962 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:34.361653 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:44787 (local address 127.30.8.62:38287)
0811 15:12:34.361805 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:34.361809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:34.362709 (+ 900us) server_negotiation.cc:407] Connection header received
0811 15:12:34.364081 (+ 1372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:34.364084 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:34.364145 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:34.364224 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:34.365843 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.366479 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.367914 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.368147 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.368230 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:34.370785 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:34.370788 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:34.370790 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:34.370819 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:34.372021 (+ 1202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:34.372566 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:34.372569 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:34.372571 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:34.372672 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:34.372959 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:34.372961 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:34.372962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:34.373148 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:34.373227 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:34.373715 (+ 488us) server_negotiation.cc:299] Negotiation successful
0811 15:12:34.373830 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 15:12:34.374850 28955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:34.362062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:38287 (local address 127.30.8.3:44787)
0811 15:12:34.362558 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:34.362607 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:34.363861 (+ 1254us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:34.364421 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:34.364431 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:34.364957 (+ 526us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:34.365667 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:34.365691 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.366601 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.366605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:34.367788 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:34.367799 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.368269 (+ 470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.368272 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:34.368424 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:34.369004 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:12:34.369022 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:34.370576 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:34.372134 (+ 1558us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:34.372142 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:34.372160 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:34.372450 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:34.372746 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:34.372749 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:34.372750 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:34.372888 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:34.373223 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:34.373228 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:34.373498 (+ 270us) client_negotiation.cc:769] Sending connection context
0811 15:12:34.373676 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 15:12:34.373965 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":123,"threads_started":1}
I20260811 15:12:34.376641 28953 heartbeater.cc:347] Connected to a master server at 127.30.8.62:38287
I20260811 15:12:34.376927 28953 heartbeater.cc:464] Registering TS with master...
I20260811 15:12:34.377498 28953 heartbeater.cc:511] Master 127.30.8.62:38287 requested a full tablet report, sending...
I20260811 15:12:34.378705 28219 ts_manager.cc:194] Registered new tserver with Master: 357c41bab8db4960b37f4d481f0678d5 (127.30.8.3:41841)
I20260811 15:12:34.379349 28219 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:44787
I20260811 15:12:34.379410 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 15:12:34.388971 28285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:34.381688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43720 (local address 127.30.8.62:38287)
0811 15:12:34.381830 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:34.381835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:34.382000 (+ 165us) server_negotiation.cc:407] Connection header received
0811 15:12:34.382109 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:34.382112 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:34.382156 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:34.382211 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:34.382851 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.383417 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.384215 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.384431 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.384504 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:34.385984 (+ 1480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:34.385987 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:34.385989 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:34.386013 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:34.387290 (+ 1277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:34.387766 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:34.387769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:34.387770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:34.387887 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:34.388157 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:34.388159 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:34.388161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:34.388412 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:34.388531 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:34.388723 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 15:12:34.388807 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 15:12:34.392141 28219 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43720:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:12:34.395326 28219 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:12:34.411170 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:34.404655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58732 (local address 127.30.8.2:42769)
0811 15:12:34.405320 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:34.405333 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:34.405707 (+ 374us) server_negotiation.cc:407] Connection header received
0811 15:12:34.405908 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:34.405914 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:34.406102 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:34.406222 (+ 120us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:34.406732 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.407896 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.409413 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.410285 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.410403 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:34.410571 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:34.410847 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 15:12:34.410984 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":567,"thread_start_us":131,"threads_started":1}
I20260811 15:12:34.411168 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:34.405137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:41841 (local address 127.0.0.1:57530)
0811 15:12:34.405588 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:34.405602 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:34.405694 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:34.406416 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:34.406419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:34.406447 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:34.406584 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:34.406590 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.408124 (+ 1534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.408127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:34.409293 (+ 1166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:34.409303 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.410413 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.410417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:34.410592 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:34.410658 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 15:12:34.410784 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 15:12:34.410972 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":383,"mutex_wait_us":175,"spinlock_wait_cycles":1408,"thread_start_us":82,"threads_started":1}
I20260811 15:12:34.411168 28967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:34.404407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:42769 (local address 127.0.0.1:58732)
0811 15:12:34.405456 (+ 1049us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:34.405634 (+ 178us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:34.405767 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:34.406422 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:34.406424 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:34.406453 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:34.406601 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:34.406609 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.408031 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.408034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:34.409287 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:34.409294 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.410413 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.410417 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:34.410587 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:34.410658 (+ 71us) client_negotiation.cc:769] Sending connection context
0811 15:12:34.410784 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 15:12:34.410972 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":990,"mutex_wait_us":100,"spinlock_wait_cycles":768,"thread_start_us":119,"threads_started":1}
I20260811 15:12:34.411938 28974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:34.405322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57530 (local address 127.30.8.3:41841)
0811 15:12:34.405938 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:34.405950 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:34.405970 (+ 20us) server_negotiation.cc:407] Connection header received
0811 15:12:34.406036 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:34.406041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:34.406273 (+ 232us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:34.406422 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:34.406705 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.407971 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.409427 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.410285 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.410403 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:34.410570 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:34.411661 (+ 1091us) server_negotiation.cc:299] Negotiation successful
0811 15:12:34.411787 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":519,"thread_start_us":112,"threads_started":1}
I20260811 15:12:34.412622 28966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:34.404325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:40279 (local address 127.0.0.1:33090)
0811 15:12:34.405137 (+ 812us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:34.405191 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:34.405329 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:34.406158 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:34.406165 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:34.406190 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:34.406314 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:34.406320 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.407575 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.407579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:34.409278 (+ 1699us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:34.409290 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.412212 (+ 2922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.412216 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:34.412306 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:34.412331 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 15:12:34.412397 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 15:12:34.412475 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"mutex_wait_us":82,"thread_start_us":88,"threads_started":1}
I20260811 15:12:34.413090 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:34.404481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33090 (local address 127.30.8.1:40279)
0811 15:12:34.405112 (+ 631us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:34.405132 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:34.405482 (+ 350us) server_negotiation.cc:407] Connection header received
0811 15:12:34.405564 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:34.405570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:34.405803 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:34.405958 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:34.406439 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.407448 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.410641 (+ 3193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.411876 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.411988 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:34.412143 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:34.412847 (+ 704us) server_negotiation.cc:299] Negotiation successful
0811 15:12:34.412962 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":499,"thread_start_us":85,"threads_started":1}
I20260811 15:12:34.415016 28630 tablet_service.cc:1467] Processing CreateTablet for tablet c42b5e5177224c979deb7430c7b4d5d9 (DEFAULT_TABLE table=test-table [id=b54bc174da314e598d9596f58a1d50c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:12:34.415019 28834 tablet_service.cc:1467] Processing CreateTablet for tablet c42b5e5177224c979deb7430c7b4d5d9 (DEFAULT_TABLE table=test-table [id=b54bc174da314e598d9596f58a1d50c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:12:34.415489 28413 tablet_service.cc:1467] Processing CreateTablet for tablet c42b5e5177224c979deb7430c7b4d5d9 (DEFAULT_TABLE table=test-table [id=b54bc174da314e598d9596f58a1d50c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:12:34.416288 28630 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c42b5e5177224c979deb7430c7b4d5d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:34.416288 28834 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c42b5e5177224c979deb7430c7b4d5d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:34.416515 28413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c42b5e5177224c979deb7430c7b4d5d9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:34.423177 28975 tablet_bootstrap.cc:492] T c42b5e5177224c979deb7430c7b4d5d9 P edffc0a281b34dd4bf02eb29e65716e3: Bootstrap starting.
I20260811 15:12:34.424253 28978 tablet_bootstrap.cc:492] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5: Bootstrap starting.
I20260811 15:12:34.425626 28975 tablet_bootstrap.cc:654] T c42b5e5177224c979deb7430c7b4d5d9 P edffc0a281b34dd4bf02eb29e65716e3: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:34.425894 28978 tablet_bootstrap.cc:654] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:34.426445 28975 log.cc:824] T c42b5e5177224c979deb7430c7b4d5d9 P edffc0a281b34dd4bf02eb29e65716e3: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:34.426959 28978 log.cc:824] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:34.427265 28977 tablet_bootstrap.cc:492] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197: Bootstrap starting.
I20260811 15:12:34.428993 28978 tablet_bootstrap.cc:492] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5: No bootstrap required, opened a new log
I20260811 15:12:34.428993 28975 tablet_bootstrap.cc:492] T c42b5e5177224c979deb7430c7b4d5d9 P edffc0a281b34dd4bf02eb29e65716e3: No bootstrap required, opened a new log
I20260811 15:12:34.429206 28975 ts_tablet_manager.cc:1397] T c42b5e5177224c979deb7430c7b4d5d9 P edffc0a281b34dd4bf02eb29e65716e3: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 15:12:34.429206 28978 ts_tablet_manager.cc:1397] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 15:12:34.429414 28977 tablet_bootstrap.cc:654] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:34.430471 28977 log.cc:824] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:34.432557 28977 tablet_bootstrap.cc:492] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197: No bootstrap required, opened a new log
I20260811 15:12:34.432761 28977 ts_tablet_manager.cc:1397] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 15:12:34.433463 28978 raft_consensus.cc:348] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edffc0a281b34dd4bf02eb29e65716e3" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40279 } } peers { permanent_uuid: "b8f9839cdbdd4d67812c2e70b7b4a197" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42769 } } peers { permanent_uuid: "357c41bab8db4960b37f4d481f0678d5" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41841 } }
I20260811 15:12:34.433593 28975 raft_consensus.cc:348] T c42b5e5177224c979deb7430c7b4d5d9 P edffc0a281b34dd4bf02eb29e65716e3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edffc0a281b34dd4bf02eb29e65716e3" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40279 } } peers { permanent_uuid: "b8f9839cdbdd4d67812c2e70b7b4a197" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42769 } } peers { permanent_uuid: "357c41bab8db4960b37f4d481f0678d5" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41841 } }
I20260811 15:12:34.433804 28975 raft_consensus.cc:374] T c42b5e5177224c979deb7430c7b4d5d9 P edffc0a281b34dd4bf02eb29e65716e3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:34.433805 28978 raft_consensus.cc:374] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:34.433863 28978 raft_consensus.cc:729] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 357c41bab8db4960b37f4d481f0678d5, State: Initialized, Role: FOLLOWER
I20260811 15:12:34.433863 28975 raft_consensus.cc:729] T c42b5e5177224c979deb7430c7b4d5d9 P edffc0a281b34dd4bf02eb29e65716e3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: edffc0a281b34dd4bf02eb29e65716e3, State: Initialized, Role: FOLLOWER
I20260811 15:12:34.434403 28978 consensus_queue.cc:260] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edffc0a281b34dd4bf02eb29e65716e3" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40279 } } peers { permanent_uuid: "b8f9839cdbdd4d67812c2e70b7b4a197" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42769 } } peers { permanent_uuid: "357c41bab8db4960b37f4d481f0678d5" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41841 } }
I20260811 15:12:34.434403 28975 consensus_queue.cc:260] T c42b5e5177224c979deb7430c7b4d5d9 P edffc0a281b34dd4bf02eb29e65716e3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edffc0a281b34dd4bf02eb29e65716e3" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40279 } } peers { permanent_uuid: "b8f9839cdbdd4d67812c2e70b7b4a197" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42769 } } peers { permanent_uuid: "357c41bab8db4960b37f4d481f0678d5" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41841 } }
I20260811 15:12:34.435554 28975 ts_tablet_manager.cc:1428] T c42b5e5177224c979deb7430c7b4d5d9 P edffc0a281b34dd4bf02eb29e65716e3: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 15:12:34.435932 28525 heartbeater.cc:503] Master 127.30.8.62:38287 was elected leader, sending a full tablet report...
I20260811 15:12:34.436260 28977 raft_consensus.cc:348] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edffc0a281b34dd4bf02eb29e65716e3" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40279 } } peers { permanent_uuid: "b8f9839cdbdd4d67812c2e70b7b4a197" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42769 } } peers { permanent_uuid: "357c41bab8db4960b37f4d481f0678d5" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41841 } }
I20260811 15:12:34.436491 28977 raft_consensus.cc:374] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:34.436592 28977 raft_consensus.cc:729] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8f9839cdbdd4d67812c2e70b7b4a197, State: Initialized, Role: FOLLOWER
I20260811 15:12:34.435784 28978 ts_tablet_manager.cc:1428] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 15:12:34.437081 28977 consensus_queue.cc:260] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edffc0a281b34dd4bf02eb29e65716e3" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40279 } } peers { permanent_uuid: "b8f9839cdbdd4d67812c2e70b7b4a197" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42769 } } peers { permanent_uuid: "357c41bab8db4960b37f4d481f0678d5" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41841 } }
I20260811 15:12:34.437243 28953 heartbeater.cc:503] Master 127.30.8.62:38287 was elected leader, sending a full tablet report...
I20260811 15:12:34.437892 28734 heartbeater.cc:503] Master 127.30.8.62:38287 was elected leader, sending a full tablet report...
I20260811 15:12:34.438134 28977 ts_tablet_manager.cc:1428] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 15:12:34.453416 28987 raft_consensus.cc:482] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:12:34.453691 28987 raft_consensus.cc:504] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edffc0a281b34dd4bf02eb29e65716e3" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40279 } } peers { permanent_uuid: "b8f9839cdbdd4d67812c2e70b7b4a197" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42769 } } peers { permanent_uuid: "357c41bab8db4960b37f4d481f0678d5" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41841 } }
I20260811 15:12:34.455044 28987 leader_election.cc:288] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers edffc0a281b34dd4bf02eb29e65716e3 (127.30.8.1:40279), 357c41bab8db4960b37f4d481f0678d5 (127.30.8.3:41841)
I20260811 15:12:34.459810 28968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:34.455384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:60907 (local address 127.30.8.1:40279)
0811 15:12:34.455541 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:34.455546 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:34.455562 (+ 16us) server_negotiation.cc:407] Connection header received
0811 15:12:34.455621 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:34.455623 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:34.455674 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:34.455752 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:34.456094 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.456869 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.458474 (+ 1605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.459146 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.459253 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:34.459324 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:34.459577 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 15:12:34.459651 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 15:12:34.459810 28974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:34.455632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:35027 (local address 127.30.8.3:41841)
0811 15:12:34.455757 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:34.455762 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:34.455918 (+ 156us) server_negotiation.cc:407] Connection header received
0811 15:12:34.456013 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:34.456015 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:34.456065 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:34.456143 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:34.456363 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.457068 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.458474 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.459140 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.459258 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:34.459324 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:34.459577 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 15:12:34.459651 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 15:12:34.459827 28736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:34.455262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:40279 (local address 127.30.8.2:60907)
0811 15:12:34.455401 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:34.455417 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:34.455514 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:34.455827 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:34.455830 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:34.455850 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:34.455975 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:34.455982 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.457008 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.457011 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:34.458315 (+ 1304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:34.458327 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.459264 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.459268 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:34.459433 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:34.459474 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 15:12:34.459554 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 15:12:34.459642 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":49}
I20260811 15:12:34.459836 28989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:34.455511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:41841 (local address 127.30.8.2:35027)
0811 15:12:34.455839 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:34.455850 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:34.455938 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:34.456163 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:34.456165 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:34.456181 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:34.456270 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:34.456274 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.457180 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.457183 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:34.458353 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:34.458360 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.459264 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.459269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:34.459415 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:34.459481 (+ 66us) client_negotiation.cc:769] Sending connection context
0811 15:12:34.459554 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 15:12:34.459642 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"mutex_wait_us":65,"thread_start_us":160,"threads_started":1}
I20260811 15:12:34.460614 28452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c42b5e5177224c979deb7430c7b4d5d9" candidate_uuid: "b8f9839cdbdd4d67812c2e70b7b4a197" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edffc0a281b34dd4bf02eb29e65716e3" is_pre_election: true
I20260811 15:12:34.460922 28871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c42b5e5177224c979deb7430c7b4d5d9" candidate_uuid: "b8f9839cdbdd4d67812c2e70b7b4a197" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "357c41bab8db4960b37f4d481f0678d5" is_pre_election: true
I20260811 15:12:34.461004 28452 raft_consensus.cc:2454] T c42b5e5177224c979deb7430c7b4d5d9 P edffc0a281b34dd4bf02eb29e65716e3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8f9839cdbdd4d67812c2e70b7b4a197 in term 0.
I20260811 15:12:34.461212 28871 raft_consensus.cc:2454] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8f9839cdbdd4d67812c2e70b7b4a197 in term 0.
I20260811 15:12:34.461541 28562 leader_election.cc:302] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b8f9839cdbdd4d67812c2e70b7b4a197, edffc0a281b34dd4bf02eb29e65716e3; no voters:
I20260811 15:12:34.461814 28987 raft_consensus.cc:2781] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:12:34.461910 28987 raft_consensus.cc:482] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:12:34.461963 28987 raft_consensus.cc:3037] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:34.463097 28987 raft_consensus.cc:504] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edffc0a281b34dd4bf02eb29e65716e3" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40279 } } peers { permanent_uuid: "b8f9839cdbdd4d67812c2e70b7b4a197" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42769 } } peers { permanent_uuid: "357c41bab8db4960b37f4d481f0678d5" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41841 } }
I20260811 15:12:34.463440 28987 leader_election.cc:288] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [CANDIDATE]: Term 1 election: Requested vote from peers edffc0a281b34dd4bf02eb29e65716e3 (127.30.8.1:40279), 357c41bab8db4960b37f4d481f0678d5 (127.30.8.3:41841)
I20260811 15:12:34.463896 28871 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c42b5e5177224c979deb7430c7b4d5d9" candidate_uuid: "b8f9839cdbdd4d67812c2e70b7b4a197" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "357c41bab8db4960b37f4d481f0678d5"
I20260811 15:12:34.463904 28452 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c42b5e5177224c979deb7430c7b4d5d9" candidate_uuid: "b8f9839cdbdd4d67812c2e70b7b4a197" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "edffc0a281b34dd4bf02eb29e65716e3"
I20260811 15:12:34.464015 28871 raft_consensus.cc:3037] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:34.464028 28452 raft_consensus.cc:3037] T c42b5e5177224c979deb7430c7b4d5d9 P edffc0a281b34dd4bf02eb29e65716e3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:34.465094 28871 raft_consensus.cc:2454] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8f9839cdbdd4d67812c2e70b7b4a197 in term 1.
I20260811 15:12:34.465130 28452 raft_consensus.cc:2454] T c42b5e5177224c979deb7430c7b4d5d9 P edffc0a281b34dd4bf02eb29e65716e3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8f9839cdbdd4d67812c2e70b7b4a197 in term 1.
I20260811 15:12:34.465413 28562 leader_election.cc:302] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 357c41bab8db4960b37f4d481f0678d5, b8f9839cdbdd4d67812c2e70b7b4a197; no voters:
I20260811 15:12:34.465610 28987 raft_consensus.cc:2781] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:34.465883 28987 raft_consensus.cc:686] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [term 1 LEADER]: Becoming Leader. State: Replica: b8f9839cdbdd4d67812c2e70b7b4a197, State: Running, Role: LEADER
I20260811 15:12:34.466287 28987 consensus_queue.cc:237] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edffc0a281b34dd4bf02eb29e65716e3" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40279 } } peers { permanent_uuid: "b8f9839cdbdd4d67812c2e70b7b4a197" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42769 } } peers { permanent_uuid: "357c41bab8db4960b37f4d481f0678d5" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41841 } }
I20260811 15:12:34.469923 28219 catalog_manager.cc:5495] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 reported cstate change: term changed from 0 to 1, leader changed from <none> to b8f9839cdbdd4d67812c2e70b7b4a197 (127.30.8.2). New cstate: current_term: 1 leader_uuid: "b8f9839cdbdd4d67812c2e70b7b4a197" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "edffc0a281b34dd4bf02eb29e65716e3" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40279 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b8f9839cdbdd4d67812c2e70b7b4a197" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "357c41bab8db4960b37f4d481f0678d5" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41841 } health_report { overall_health: UNKNOWN } } }
I20260811 15:12:34.491962 28969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:34.488707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58734 (local address 127.30.8.2:42769)
0811 15:12:34.488837 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:34.488841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:34.488854 (+ 13us) server_negotiation.cc:407] Connection header received
0811 15:12:34.488930 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:34.488932 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:34.488988 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:34.489079 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:12:34.489266 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.489763 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.490573 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.490778 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.490847 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:34.491136 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:12:34.491575 (+ 439us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:12:34.491698 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:34.491772 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 15:12:34.491812 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 15:12:34.498195 28452 raft_consensus.cc:1261] T c42b5e5177224c979deb7430c7b4d5d9 P edffc0a281b34dd4bf02eb29e65716e3 [term 1 FOLLOWER]: Refusing update from remote peer b8f9839cdbdd4d67812c2e70b7b4a197: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:12:34.498206 28871 raft_consensus.cc:1261] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5 [term 1 FOLLOWER]: Refusing update from remote peer b8f9839cdbdd4d67812c2e70b7b4a197: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:12:34.498950 28987 consensus_queue.cc:1035] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [LEADER]: Connected to new peer: Peer: permanent_uuid: "357c41bab8db4960b37f4d481f0678d5" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 41841 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:12:34.499234 28990 consensus_queue.cc:1035] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [LEADER]: Connected to new peer: Peer: permanent_uuid: "edffc0a281b34dd4bf02eb29e65716e3" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 40279 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:12:34.510196 28995 mvcc.cc:205] Tried to move back new op lower bound from 7317344888816795648 to 7317344888697204736. Current Snapshot: MvccSnapshot[applied={T|T < 7317344888816795648}]
I20260811 15:12:34.516489 28217 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:12:34.516680 28217 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43720:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:12:34.519266 28217 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fdb4552d922348bea9d05264160d501f: Sending DeleteTablet for 3 replicas of tablet c42b5e5177224c979deb7430c7b4d5d9
I20260811 15:12:34.520184 28630 tablet_service.cc:1514] Processing DeleteTablet for tablet c42b5e5177224c979deb7430c7b4d5d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:12:34 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:58732
I20260811 15:12:34.520293 28413 tablet_service.cc:1514] Processing DeleteTablet for tablet c42b5e5177224c979deb7430c7b4d5d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:12:34 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:33090
I20260811 15:12:34.520490 28834 tablet_service.cc:1514] Processing DeleteTablet for tablet c42b5e5177224c979deb7430c7b4d5d9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:12:34 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:57530
I20260811 15:12:34.520771 29008 tablet_replica.cc:330] T c42b5e5177224c979deb7430c7b4d5d9 P edffc0a281b34dd4bf02eb29e65716e3: stopping tablet replica
I20260811 15:12:34.520773 29007 tablet_replica.cc:330] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197: stopping tablet replica
I20260811 15:12:34.521121 29008 raft_consensus.cc:2229] T c42b5e5177224c979deb7430c7b4d5d9 P edffc0a281b34dd4bf02eb29e65716e3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:12:34.521121 29007 raft_consensus.cc:2229] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:12:34.521500 29008 raft_consensus.cc:2258] T c42b5e5177224c979deb7430c7b4d5d9 P edffc0a281b34dd4bf02eb29e65716e3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:12:34.521520 29007 raft_consensus.cc:2258] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:12:34.521910 29009 tablet_replica.cc:330] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5: stopping tablet replica
I20260811 15:12:34.522076 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 28319
I20260811 15:12:34.522193 29009 raft_consensus.cc:2229] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:12:34.522440 29009 raft_consensus.cc:2258] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:12:34.535126 29007 ts_tablet_manager.cc:1905] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:12:34.538595 29007 ts_tablet_manager.cc:1918] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:12:34.538820 29007 log.cc:1196] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/c42b5e5177224c979deb7430c7b4d5d9
I20260811 15:12:34.539280 29007 ts_tablet_manager.cc:1939] T c42b5e5177224c979deb7430c7b4d5d9 P b8f9839cdbdd4d67812c2e70b7b4a197: Deleting consensus metadata
I20260811 15:12:34.540299 29009 ts_tablet_manager.cc:1905] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:12:34.541107 28197 catalog_manager.cc:4841] TS b8f9839cdbdd4d67812c2e70b7b4a197 (127.30.8.2:42769): tablet c42b5e5177224c979deb7430c7b4d5d9 (table test-table [id=b54bc174da314e598d9596f58a1d50c1]) successfully deleted
I20260811 15:12:34.543298 29009 ts_tablet_manager.cc:1918] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:12:34.543418 29009 log.cc:1196] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/c42b5e5177224c979deb7430c7b4d5d9
I20260811 15:12:34.543776 29009 ts_tablet_manager.cc:1939] T c42b5e5177224c979deb7430c7b4d5d9 P 357c41bab8db4960b37f4d481f0678d5: Deleting consensus metadata
W20260811 15:12:34.545254 28204 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.8.1:40279 (error 108)
I20260811 15:12:34.545784 28204 catalog_manager.cc:4841] TS 357c41bab8db4960b37f4d481f0678d5 (127.30.8.3:41841): tablet c42b5e5177224c979deb7430c7b4d5d9 (table test-table [id=b54bc174da314e598d9596f58a1d50c1]) successfully deleted
I20260811 15:12:34.546352 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 28533
I20260811 15:12:34.546424 28966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:34.546091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:40279 (local address 127.0.0.1:33106)
0811 15:12:34.546245 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:34.546329 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.1:40279: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 15:12:34.568794 28204 catalog_manager.cc:4584] TS edffc0a281b34dd4bf02eb29e65716e3 (127.30.8.1:40279): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c42b5e5177224c979deb7430c7b4d5d9: Network error: Client connection negotiation failed: client connection to 127.30.8.1:40279: connect: Connection refused (error 111)
I20260811 15:12:34.574682 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 28741
I20260811 15:12:34.592510 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 28169
2026-08-11T15:12:34Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2520 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T15:12:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:12:34Z Disabled control of system clock
I20260811 15:12:34.638669 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:43167
--webserver_interface=127.30.8.62
--webserver_port=0
--builtin_ntp_servers=127.30.8.20:36385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:43167
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 15:12:34.756827 29016 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:34.757153 29016 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:34.757193 29016 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:34.761540 29016 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:12:34.761641 29016 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:34.761655 29016 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:34.761672 29016 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:12:34.761694 29016 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:12:34.767007 29016 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:43167
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:43167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29016
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:34.768256 29016 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:34.769745 29016 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:34.778600 29024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:34.778635 29016 server_base.cc:1034] running on GCE node
W20260811 15:12:34.778848 29027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:34.778713 29023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:34.779604 29016 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:34.780855 29016 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:34.782032 29016 hybrid_clock.cc:648] HybridClock initialized: now 1786461154782020 us; error 31 us; skew 500 ppm
I20260811 15:12:34.784118 29016 webserver.cc:460] Webserver started at http://127.30.8.62:45585/ using document root <none> and password file <none>
I20260811 15:12:34.784755 29016 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:34.784835 29016 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:34.785058 29016 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:34.787352 29016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "41941b9b1c3147bcb83580c4a15a0270"
format_stamp: "Formatted at 2026-08-11 15:12:34 on dist-test-slave-1vmg"
server_key: "d7a020d1b00fea68d97437f3f86c636f"
server_key_iv: "1a95208ce04bf6cebaaf08e82304363b"
server_key_version: "encryptionkey@0"
I20260811 15:12:34.787961 29016 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "41941b9b1c3147bcb83580c4a15a0270"
format_stamp: "Formatted at 2026-08-11 15:12:34 on dist-test-slave-1vmg"
server_key: "d7a020d1b00fea68d97437f3f86c636f"
server_key_iv: "1a95208ce04bf6cebaaf08e82304363b"
server_key_version: "encryptionkey@0"
I20260811 15:12:34.792055 29016 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:12:34.794754 29036 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:34.796146 29016 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:12:34.796289 29016 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "41941b9b1c3147bcb83580c4a15a0270"
format_stamp: "Formatted at 2026-08-11 15:12:34 on dist-test-slave-1vmg"
server_key: "d7a020d1b00fea68d97437f3f86c636f"
server_key_iv: "1a95208ce04bf6cebaaf08e82304363b"
server_key_version: "encryptionkey@0"
I20260811 15:12:34.796409 29016 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:34.828167 29016 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:34.828956 29016 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:34.829177 29016 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:34.836558 29016 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:43167
I20260811 15:12:34.836582 29121 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:43167 every 8 connection(s)
I20260811 15:12:34.837620 29016 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:12:34.841167 29122 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:34.846107 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 29016
I20260811 15:12:34.846252 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:12:34.846499 30752 external_mini_cluster.cc:1428] Setting key fd8a0afb9a25c042f35e1dd9d2464945
I20260811 15:12:34.847658 29122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270: Bootstrap starting.
I20260811 15:12:34.850273 29122 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:34.851130 29122 log.cc:824] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:34.853217 29122 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270: No bootstrap required, opened a new log
I20260811 15:12:34.856212 29127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:34.848064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48122 (local address 127.30.8.62:43167)
0811 15:12:34.848794 (+ 730us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:34.848811 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:34.848849 (+ 38us) server_negotiation.cc:407] Connection header received
0811 15:12:34.849540 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:34.849572 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:34.850011 (+ 439us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:34.850343 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:34.850832 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.851914 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.852812 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:34.853298 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:34.853453 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:34.854320 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:34.854326 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:34.854341 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:34.854763 (+ 422us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:34.854825 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:34.854846 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:34.854959 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:34.855120 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 15:12:34.855415 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":154,"threads_started":1}
I20260811 15:12:34.857374 29122 raft_consensus.cc:348] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41941b9b1c3147bcb83580c4a15a0270" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43167 } }
I20260811 15:12:34.857630 29122 raft_consensus.cc:374] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:34.857692 29122 raft_consensus.cc:729] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41941b9b1c3147bcb83580c4a15a0270, State: Initialized, Role: FOLLOWER
I20260811 15:12:34.858156 29122 consensus_queue.cc:260] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41941b9b1c3147bcb83580c4a15a0270" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43167 } }
I20260811 15:12:34.858307 29122 raft_consensus.cc:388] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:12:34.858378 29122 raft_consensus.cc:482] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:12:34.858489 29122 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:34.859596 29122 raft_consensus.cc:504] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41941b9b1c3147bcb83580c4a15a0270" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43167 } }
I20260811 15:12:34.860020 29122 leader_election.cc:302] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 41941b9b1c3147bcb83580c4a15a0270; no voters:
I20260811 15:12:34.860378 29122 leader_election.cc:288] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:12:34.860494 29129 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:34.860754 29129 raft_consensus.cc:686] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270 [term 1 LEADER]: Becoming Leader. State: Replica: 41941b9b1c3147bcb83580c4a15a0270, State: Running, Role: LEADER
I20260811 15:12:34.861083 29129 consensus_queue.cc:237] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41941b9b1c3147bcb83580c4a15a0270" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43167 } }
I20260811 15:12:34.861696 29122 sys_catalog.cc:564] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:12:34.862845 29130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "41941b9b1c3147bcb83580c4a15a0270" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41941b9b1c3147bcb83580c4a15a0270" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43167 } } }
I20260811 15:12:34.863014 29130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270 [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:34.863287 29131 sys_catalog.cc:455] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 41941b9b1c3147bcb83580c4a15a0270. Latest consensus state: current_term: 1 leader_uuid: "41941b9b1c3147bcb83580c4a15a0270" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "41941b9b1c3147bcb83580c4a15a0270" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 43167 } } }
I20260811 15:12:34.863400 29131 sys_catalog.cc:458] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270 [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:34.863502 29141 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:12:34.866575 29141 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:12:34.872421 29141 catalog_manager.cc:1347] Generated new cluster ID: 087142421c6b46c1a4a081a7f678f317
I20260811 15:12:34.872525 29141 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:12:34.895495 29141 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:12:34.896566 29141 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:12:34.902130 29141 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270: Generated new TSK 0
I20260811 15:12:34.903141 29141 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:12:34.914129 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:43167
--builtin_ntp_servers=127.30.8.20:36385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 15:12:35.037153 29156 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:35.037483 29156 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:35.037523 29156 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:35.041795 29156 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:35.041882 29156 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:35.041954 29156 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:12:35.047143 29156 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:43167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29156
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:35.048415 29156 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:35.049887 29156 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:35.060124 29168 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:35.060133 29164 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:35.060226 29165 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:35.060395 29156 server_base.cc:1034] running on GCE node
I20260811 15:12:35.060891 29156 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:35.061657 29156 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:35.062848 29156 hybrid_clock.cc:648] HybridClock initialized: now 1786461155062827 us; error 34 us; skew 500 ppm
I20260811 15:12:35.064996 29156 webserver.cc:460] Webserver started at http://127.30.8.1:44239/ using document root <none> and password file <none>
I20260811 15:12:35.065619 29156 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:35.065698 29156 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:35.065938 29156 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:35.068264 29156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "eb7d8eb084194232bfdb4f1e804c8bb9"
format_stamp: "Formatted at 2026-08-11 15:12:35 on dist-test-slave-1vmg"
server_key: "fc45990131c6147fc525a523c7ab582d"
server_key_iv: "92f00dd9132a0a3aac793da55c7d0c8c"
server_key_version: "encryptionkey@0"
I20260811 15:12:35.068879 29156 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "eb7d8eb084194232bfdb4f1e804c8bb9"
format_stamp: "Formatted at 2026-08-11 15:12:35 on dist-test-slave-1vmg"
server_key: "fc45990131c6147fc525a523c7ab582d"
server_key_iv: "92f00dd9132a0a3aac793da55c7d0c8c"
server_key_version: "encryptionkey@0"
I20260811 15:12:35.073058 29156 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:12:35.075829 29178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:35.077165 29156 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:12:35.077306 29156 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "eb7d8eb084194232bfdb4f1e804c8bb9"
format_stamp: "Formatted at 2026-08-11 15:12:35 on dist-test-slave-1vmg"
server_key: "fc45990131c6147fc525a523c7ab582d"
server_key_iv: "92f00dd9132a0a3aac793da55c7d0c8c"
server_key_version: "encryptionkey@0"
I20260811 15:12:35.077433 29156 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:35.090569 29156 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:35.091454 29156 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:35.091660 29156 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:35.092377 29156 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:35.093454 29156 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:12:35.093508 29156 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:35.093567 29156 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:12:35.093596 29156 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:35.102849 29156 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:45715
I20260811 15:12:35.102871 29361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:45715 every 8 connection(s)
I20260811 15:12:35.103890 29156 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:12:35.112644 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 29156
I20260811 15:12:35.112800 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:12:35.113071 30752 external_mini_cluster.cc:1428] Setting key d66fb32b1bec3e55ef0f8f09ed817207
I20260811 15:12:35.113613 29127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:35.105849 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:42393 (local address 127.30.8.62:43167)
0811 15:12:35.106004 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:35.106009 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:35.107251 (+ 1242us) server_negotiation.cc:407] Connection header received
0811 15:12:35.108660 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:35.108664 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:35.108714 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:35.108793 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:35.109761 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.110295 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.111535 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.111775 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.111861 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:35.113062 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:35.113065 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:35.113068 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:35.113083 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:35.113105 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:35.113114 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:35.113176 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:35.113381 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 15:12:35.113458 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 15:12:35.114639 29368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:35.106309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:43167 (local address 127.30.8.1:42393)
0811 15:12:35.107086 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:35.107132 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:35.108441 (+ 1309us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:35.108972 (+ 531us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:35.108978 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:35.109070 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:35.109633 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:35.109649 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.110400 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.110402 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:35.111412 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:35.111424 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.111913 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.111916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:35.112068 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:35.112722 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:12:35.112724 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:35.112773 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:12:35.112878 (+ 105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:35.113222 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:35.113233 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:35.113238 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 15:12:35.113361 (+ 123us) client_negotiation.cc:240] Negotiation successful
0811 15:12:35.113720 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":125,"threads_started":1}
I20260811 15:12:35.115149 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:43167
--builtin_ntp_servers=127.30.8.20:36385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 15:12:35.116016 29363 heartbeater.cc:347] Connected to a master server at 127.30.8.62:43167
I20260811 15:12:35.116286 29363 heartbeater.cc:464] Registering TS with master...
I20260811 15:12:35.116866 29363 heartbeater.cc:511] Master 127.30.8.62:43167 requested a full tablet report, sending...
I20260811 15:12:35.118454 29064 ts_manager.cc:194] Registered new tserver with Master: eb7d8eb084194232bfdb4f1e804c8bb9 (127.30.8.1:45715)
I20260811 15:12:35.119709 29064 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.8.1:42393
W20260811 15:12:35.234688 29369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:35.235074 29369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:35.235142 29369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:35.239326 29369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:35.239408 29369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:35.239490 29369 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:12:35.244607 29369 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:43167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:35.245848 29369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:35.247318 29369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:35.257337 29386 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:35.257359 29381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:35.257359 29384 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:35.258002 29369 server_base.cc:1034] running on GCE node
I20260811 15:12:35.258430 29369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:35.259177 29369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:35.260337 29369 hybrid_clock.cc:648] HybridClock initialized: now 1786461155260311 us; error 42 us; skew 500 ppm
I20260811 15:12:35.262434 29369 webserver.cc:460] Webserver started at http://127.30.8.2:36681/ using document root <none> and password file <none>
I20260811 15:12:35.263130 29369 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:35.263208 29369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:35.263437 29369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:35.265699 29369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "0b98a385304b40c28a9b87b291f378a8"
format_stamp: "Formatted at 2026-08-11 15:12:35 on dist-test-slave-1vmg"
server_key: "1c27047c56b20c298208eec9e89489db"
server_key_iv: "16e58372f342cd91813b7289151bd51f"
server_key_version: "encryptionkey@0"
I20260811 15:12:35.266278 29369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "0b98a385304b40c28a9b87b291f378a8"
format_stamp: "Formatted at 2026-08-11 15:12:35 on dist-test-slave-1vmg"
server_key: "1c27047c56b20c298208eec9e89489db"
server_key_iv: "16e58372f342cd91813b7289151bd51f"
server_key_version: "encryptionkey@0"
I20260811 15:12:35.270288 29369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 15:12:35.272969 29394 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:35.274288 29369 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 15:12:35.274422 29369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "0b98a385304b40c28a9b87b291f378a8"
format_stamp: "Formatted at 2026-08-11 15:12:35 on dist-test-slave-1vmg"
server_key: "1c27047c56b20c298208eec9e89489db"
server_key_iv: "16e58372f342cd91813b7289151bd51f"
server_key_version: "encryptionkey@0"
I20260811 15:12:35.274541 29369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:35.296979 29369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:35.297777 29369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:35.297972 29369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:35.298620 29369 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:35.299751 29369 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:12:35.299808 29369 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:35.299872 29369 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:12:35.299897 29369 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:35.308979 29369 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:44329
I20260811 15:12:35.309001 29574 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:44329 every 8 connection(s)
I20260811 15:12:35.309962 29369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:12:35.313179 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 29369
I20260811 15:12:35.313342 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:12:35.313671 30752 external_mini_cluster.cc:1428] Setting key 360d2e567c982603a822c4e3c2bea3f1
I20260811 15:12:35.315706 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:43167
--builtin_ntp_servers=127.30.8.20:36385
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 15:12:35.319854 29127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:35.311952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:50793 (local address 127.30.8.62:43167)
0811 15:12:35.312108 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:35.312112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:35.313329 (+ 1217us) server_negotiation.cc:407] Connection header received
0811 15:12:35.314968 (+ 1639us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:35.314971 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:35.315028 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:35.315105 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:35.316119 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.316652 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.317837 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.318064 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.318159 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:35.319336 (+ 1177us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:35.319339 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:35.319341 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:35.319356 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:35.319380 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:35.319388 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:35.319448 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:35.319606 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 15:12:35.319679 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 15:12:35.320854 29578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:35.312368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:43167 (local address 127.30.8.2:50793)
0811 15:12:35.313139 (+ 771us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:35.313200 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:35.314712 (+ 1512us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:35.315306 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:35.315313 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:35.315405 (+ 92us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:35.315974 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:35.315990 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.316793 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.316796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:35.317699 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:35.317721 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.318225 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.318230 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:35.318400 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:35.319054 (+ 654us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:12:35.319055 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:35.319099 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:12:35.319175 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:35.319461 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:35.319472 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:35.319478 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 15:12:35.319599 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 15:12:35.319948 (+ 349us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":136,"threads_started":1}
I20260811 15:12:35.322252 29575 heartbeater.cc:347] Connected to a master server at 127.30.8.62:43167
I20260811 15:12:35.322494 29575 heartbeater.cc:464] Registering TS with master...
I20260811 15:12:35.323103 29575 heartbeater.cc:511] Master 127.30.8.62:43167 requested a full tablet report, sending...
I20260811 15:12:35.324321 29064 ts_manager.cc:194] Registered new tserver with Master: 0b98a385304b40c28a9b87b291f378a8 (127.30.8.2:44329)
I20260811 15:12:35.325130 29064 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.8.2:50793
W20260811 15:12:35.436844 29580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:35.437148 29580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:35.437187 29580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:35.441421 29580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:35.441504 29580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:35.441566 29580 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:12:35.446568 29580 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:36385
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:43167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:35.447791 29580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:35.449172 29580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:35.459203 29589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:35.459153 29592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:35.459244 29580 server_base.cc:1034] running on GCE node
W20260811 15:12:35.459493 29590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:35.459949 29580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:35.460618 29580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:35.461786 29580 hybrid_clock.cc:648] HybridClock initialized: now 1786461155461761 us; error 35 us; skew 500 ppm
I20260811 15:12:35.463958 29580 webserver.cc:460] Webserver started at http://127.30.8.3:45667/ using document root <none> and password file <none>
I20260811 15:12:35.464553 29580 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:35.464615 29580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:35.464807 29580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:35.466959 29580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "f3707319f2e041e1aa4d3d1ff0fcd965"
format_stamp: "Formatted at 2026-08-11 15:12:35 on dist-test-slave-1vmg"
server_key: "241088bc4be5e9bfe8b18bbfc44530e8"
server_key_iv: "f8db5408c0871ae900f0c92f548c6258"
server_key_version: "encryptionkey@0"
I20260811 15:12:35.467535 29580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "f3707319f2e041e1aa4d3d1ff0fcd965"
format_stamp: "Formatted at 2026-08-11 15:12:35 on dist-test-slave-1vmg"
server_key: "241088bc4be5e9bfe8b18bbfc44530e8"
server_key_iv: "f8db5408c0871ae900f0c92f548c6258"
server_key_version: "encryptionkey@0"
I20260811 15:12:35.471459 29580 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 15:12:35.474059 29604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:35.475363 29580 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:12:35.475481 29580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "f3707319f2e041e1aa4d3d1ff0fcd965"
format_stamp: "Formatted at 2026-08-11 15:12:35 on dist-test-slave-1vmg"
server_key: "241088bc4be5e9bfe8b18bbfc44530e8"
server_key_iv: "f8db5408c0871ae900f0c92f548c6258"
server_key_version: "encryptionkey@0"
I20260811 15:12:35.475593 29580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:35.485425 29580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:35.486172 29580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:35.486349 29580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:35.487064 29580 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:35.488117 29580 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:12:35.488175 29580 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:35.488221 29580 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:12:35.488250 29580 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:35.497294 29580 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:39059
I20260811 15:12:35.497319 29776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:39059 every 8 connection(s)
I20260811 15:12:35.498253 29580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:12:35.503679 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 29580
I20260811 15:12:35.503799 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:12:35.504055 30752 external_mini_cluster.cc:1428] Setting key 0e3aa29661cfc395c29ba195ee6f1ac2
I20260811 15:12:35.507864 29127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:35.500148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:41187 (local address 127.30.8.62:43167)
0811 15:12:35.500282 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:35.500286 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:35.501454 (+ 1168us) server_negotiation.cc:407] Connection header received
0811 15:12:35.502907 (+ 1453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:35.502941 (+ 34us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:35.503017 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:35.503093 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:35.504135 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.504732 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.505910 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.506146 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.506238 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:35.507368 (+ 1130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:35.507371 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:35.507373 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:35.507386 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:35.507409 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:35.507417 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:35.507478 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:35.507647 (+ 169us) server_negotiation.cc:299] Negotiation successful
0811 15:12:35.507713 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 15:12:35.508781 29784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:35.500589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:43167 (local address 127.30.8.3:41187)
0811 15:12:35.501310 (+ 721us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:35.501358 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:35.502680 (+ 1322us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:35.503316 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:35.503324 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:35.503419 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:35.503986 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:35.504002 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.504864 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.504867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:35.505782 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:35.505794 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.506303 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.506308 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:35.506473 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:35.507093 (+ 620us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:12:35.507094 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:35.507138 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:12:35.507211 (+ 73us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:35.507489 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:35.507509 (+ 20us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:35.507515 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 15:12:35.507631 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 15:12:35.507969 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":127,"threads_started":1}
I20260811 15:12:35.509841 29777 heartbeater.cc:347] Connected to a master server at 127.30.8.62:43167
I20260811 15:12:35.510077 29777 heartbeater.cc:464] Registering TS with master...
I20260811 15:12:35.510653 29777 heartbeater.cc:511] Master 127.30.8.62:43167 requested a full tablet report, sending...
I20260811 15:12:35.511729 29064 ts_manager.cc:194] Registered new tserver with Master: f3707319f2e041e1aa4d3d1ff0fcd965 (127.30.8.3:39059)
I20260811 15:12:35.512296 29064 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.8.3:41187
I20260811 15:12:35.517817 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 15:12:35.523604 29127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:35.519787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48128 (local address 127.30.8.62:43167)
0811 15:12:35.519913 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:35.519918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:35.520077 (+ 159us) server_negotiation.cc:407] Connection header received
0811 15:12:35.520223 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:35.520227 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:35.520273 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:35.520334 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:35.520609 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.521140 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.521958 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.522182 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.522270 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:35.523105 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:35.523109 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:35.523112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:35.523126 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:35.523147 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:35.523155 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:35.523266 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:35.523406 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 15:12:35.523449 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 15:12:35.526607 29064 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48128:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:12:35.529799 29064 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:12:35.544775 29797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:35.538467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:44329 (local address 127.0.0.1:51144)
0811 15:12:35.538847 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:35.538904 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:35.539066 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:35.540772 (+ 1706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:35.540775 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:35.540812 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:35.540927 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:35.540933 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.542090 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.542093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:35.543371 (+ 1278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:35.543378 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.544184 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.544188 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:35.544295 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:35.544331 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 15:12:35.544446 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 15:12:35.544582 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":103,"spinlock_wait_cycles":7808,"thread_start_us":106,"threads_started":1}
I20260811 15:12:35.544764 29800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:35.538617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51144 (local address 127.30.8.2:44329)
0811 15:12:35.540365 (+ 1748us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:35.540376 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:35.540398 (+ 22us) server_negotiation.cc:407] Connection header received
0811 15:12:35.540465 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:35.540469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:35.540623 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:35.540784 (+ 161us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:35.541045 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.541962 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.543506 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.544061 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.544171 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:35.544399 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:35.544477 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 15:12:35.544582 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1644,"thread_start_us":106,"threads_started":1}
I20260811 15:12:35.544814 29801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:35.538708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:39059 (local address 127.0.0.1:33694)
0811 15:12:35.539382 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:35.539395 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:35.539486 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:35.540250 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:35.540256 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:35.540280 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:35.540420 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:35.540426 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.541753 (+ 1327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.541757 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:35.543304 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:35.543314 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.544388 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.544392 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:35.544492 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:35.544516 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 15:12:35.544566 (+ 50us) client_negotiation.cc:240] Negotiation successful
0811 15:12:35.544651 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"mutex_wait_us":55,"thread_start_us":116,"threads_started":1}
I20260811 15:12:35.545104 29805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:35.539401 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48466 (local address 127.30.8.1:45715)
0811 15:12:35.539754 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:35.539766 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:35.540137 (+ 371us) server_negotiation.cc:407] Connection header received
0811 15:12:35.540298 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:35.540304 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:35.540478 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:35.540593 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:35.541102 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.542191 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.543470 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.544245 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.544358 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:35.544550 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:35.544844 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 15:12:35.544955 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":239,"thread_start_us":156,"threads_started":1}
I20260811 15:12:35.545455 29806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:35.539805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:45715 (local address 127.0.0.1:48466)
0811 15:12:35.540031 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:35.540042 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:35.540143 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:35.540780 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:35.540783 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:35.540814 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:35.540937 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:35.540944 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.542313 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.542316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:35.543361 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:35.543373 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.544610 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.544614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:35.544689 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:35.544710 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 15:12:35.545255 (+ 545us) client_negotiation.cc:240] Negotiation successful
0811 15:12:35.545335 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":110,"spinlock_wait_cycles":3072,"thread_start_us":83,"threads_started":1}
I20260811 15:12:35.545871 29802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:35.539342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33694 (local address 127.30.8.3:39059)
0811 15:12:35.539827 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:35.539839 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:35.539865 (+ 26us) server_negotiation.cc:407] Connection header received
0811 15:12:35.539929 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:35.539933 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:35.540093 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:35.540239 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:35.540534 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.541648 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.543419 (+ 1771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.544245 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.545411 (+ 1166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:35.545602 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:35.545659 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 15:12:35.545740 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":372,"thread_start_us":162,"threads_started":1}
I20260811 15:12:35.548097 29465 tablet_service.cc:1467] Processing CreateTablet for tablet e576771abe7f472895f636bc3321dcc8 (DEFAULT_TABLE table=test-table [id=65d69ab4137f4e4997b3e1c59b65c965]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:12:35.548540 29680 tablet_service.cc:1467] Processing CreateTablet for tablet e576771abe7f472895f636bc3321dcc8 (DEFAULT_TABLE table=test-table [id=65d69ab4137f4e4997b3e1c59b65c965]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:12:35.548538 29247 tablet_service.cc:1467] Processing CreateTablet for tablet e576771abe7f472895f636bc3321dcc8 (DEFAULT_TABLE table=test-table [id=65d69ab4137f4e4997b3e1c59b65c965]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:12:35.549212 29465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e576771abe7f472895f636bc3321dcc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:35.549526 29247 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e576771abe7f472895f636bc3321dcc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:35.549526 29680 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e576771abe7f472895f636bc3321dcc8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:35.556133 29808 tablet_bootstrap.cc:492] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8: Bootstrap starting.
I20260811 15:12:35.557231 29809 tablet_bootstrap.cc:492] T e576771abe7f472895f636bc3321dcc8 P eb7d8eb084194232bfdb4f1e804c8bb9: Bootstrap starting.
I20260811 15:12:35.558652 29808 tablet_bootstrap.cc:654] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:35.558876 29809 tablet_bootstrap.cc:654] T e576771abe7f472895f636bc3321dcc8 P eb7d8eb084194232bfdb4f1e804c8bb9: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:35.559315 29810 tablet_bootstrap.cc:492] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965: Bootstrap starting.
I20260811 15:12:35.559506 29808 log.cc:824] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:35.559762 29809 log.cc:824] T e576771abe7f472895f636bc3321dcc8 P eb7d8eb084194232bfdb4f1e804c8bb9: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:35.561582 29810 tablet_bootstrap.cc:654] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:35.561837 29808 tablet_bootstrap.cc:492] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8: No bootstrap required, opened a new log
I20260811 15:12:35.562032 29809 tablet_bootstrap.cc:492] T e576771abe7f472895f636bc3321dcc8 P eb7d8eb084194232bfdb4f1e804c8bb9: No bootstrap required, opened a new log
I20260811 15:12:35.562050 29808 ts_tablet_manager.cc:1397] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 15:12:35.562239 29809 ts_tablet_manager.cc:1397] T e576771abe7f472895f636bc3321dcc8 P eb7d8eb084194232bfdb4f1e804c8bb9: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 15:12:35.562506 29810 log.cc:824] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:35.564735 29810 tablet_bootstrap.cc:492] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965: No bootstrap required, opened a new log
I20260811 15:12:35.564932 29810 ts_tablet_manager.cc:1397] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 15:12:35.565857 29809 raft_consensus.cc:348] T e576771abe7f472895f636bc3321dcc8 P eb7d8eb084194232bfdb4f1e804c8bb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b98a385304b40c28a9b87b291f378a8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44329 } } peers { permanent_uuid: "eb7d8eb084194232bfdb4f1e804c8bb9" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45715 } } peers { permanent_uuid: "f3707319f2e041e1aa4d3d1ff0fcd965" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 39059 } }
I20260811 15:12:35.566093 29809 raft_consensus.cc:374] T e576771abe7f472895f636bc3321dcc8 P eb7d8eb084194232bfdb4f1e804c8bb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:35.566154 29809 raft_consensus.cc:729] T e576771abe7f472895f636bc3321dcc8 P eb7d8eb084194232bfdb4f1e804c8bb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb7d8eb084194232bfdb4f1e804c8bb9, State: Initialized, Role: FOLLOWER
I20260811 15:12:35.566340 29808 raft_consensus.cc:348] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b98a385304b40c28a9b87b291f378a8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44329 } } peers { permanent_uuid: "eb7d8eb084194232bfdb4f1e804c8bb9" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45715 } } peers { permanent_uuid: "f3707319f2e041e1aa4d3d1ff0fcd965" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 39059 } }
I20260811 15:12:35.566581 29808 raft_consensus.cc:374] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:35.566633 29808 raft_consensus.cc:729] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0b98a385304b40c28a9b87b291f378a8, State: Initialized, Role: FOLLOWER
I20260811 15:12:35.566605 29809 consensus_queue.cc:260] T e576771abe7f472895f636bc3321dcc8 P eb7d8eb084194232bfdb4f1e804c8bb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b98a385304b40c28a9b87b291f378a8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44329 } } peers { permanent_uuid: "eb7d8eb084194232bfdb4f1e804c8bb9" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45715 } } peers { permanent_uuid: "f3707319f2e041e1aa4d3d1ff0fcd965" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 39059 } }
I20260811 15:12:35.567085 29808 consensus_queue.cc:260] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b98a385304b40c28a9b87b291f378a8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44329 } } peers { permanent_uuid: "eb7d8eb084194232bfdb4f1e804c8bb9" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45715 } } peers { permanent_uuid: "f3707319f2e041e1aa4d3d1ff0fcd965" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 39059 } }
I20260811 15:12:35.567414 29363 heartbeater.cc:503] Master 127.30.8.62:43167 was elected leader, sending a full tablet report...
I20260811 15:12:35.567714 29809 ts_tablet_manager.cc:1428] T e576771abe7f472895f636bc3321dcc8 P eb7d8eb084194232bfdb4f1e804c8bb9: Time spent starting tablet: real 0.005s user 0.002s sys 0.003s
I20260811 15:12:35.568005 29808 ts_tablet_manager.cc:1428] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 15:12:35.568316 29575 heartbeater.cc:503] Master 127.30.8.62:43167 was elected leader, sending a full tablet report...
I20260811 15:12:35.570977 29810 raft_consensus.cc:348] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b98a385304b40c28a9b87b291f378a8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44329 } } peers { permanent_uuid: "eb7d8eb084194232bfdb4f1e804c8bb9" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45715 } } peers { permanent_uuid: "f3707319f2e041e1aa4d3d1ff0fcd965" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 39059 } }
I20260811 15:12:35.571288 29810 raft_consensus.cc:374] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:35.571352 29810 raft_consensus.cc:729] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f3707319f2e041e1aa4d3d1ff0fcd965, State: Initialized, Role: FOLLOWER
I20260811 15:12:35.571861 29810 consensus_queue.cc:260] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b98a385304b40c28a9b87b291f378a8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44329 } } peers { permanent_uuid: "eb7d8eb084194232bfdb4f1e804c8bb9" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45715 } } peers { permanent_uuid: "f3707319f2e041e1aa4d3d1ff0fcd965" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 39059 } }
I20260811 15:12:35.572665 29777 heartbeater.cc:503] Master 127.30.8.62:43167 was elected leader, sending a full tablet report...
I20260811 15:12:35.572985 29810 ts_tablet_manager.cc:1428] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
W20260811 15:12:35.605147 29365 tablet.cc:2369] T e576771abe7f472895f636bc3321dcc8 P eb7d8eb084194232bfdb4f1e804c8bb9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 15:12:35.749337 29781 tablet.cc:2369] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 15:12:35.811252 29576 tablet.cc:2369] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:12:35.819856 29818 raft_consensus.cc:482] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:12:35.820089 29818 raft_consensus.cc:504] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b98a385304b40c28a9b87b291f378a8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44329 } } peers { permanent_uuid: "eb7d8eb084194232bfdb4f1e804c8bb9" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45715 } } peers { permanent_uuid: "f3707319f2e041e1aa4d3d1ff0fcd965" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 39059 } }
I20260811 15:12:35.821329 29818 leader_election.cc:288] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb7d8eb084194232bfdb4f1e804c8bb9 (127.30.8.1:45715), f3707319f2e041e1aa4d3d1ff0fcd965 (127.30.8.3:39059)
I20260811 15:12:35.826486 29802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:35.821730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:36937 (local address 127.30.8.3:39059)
0811 15:12:35.821924 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:35.821928 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:35.822015 (+ 87us) server_negotiation.cc:407] Connection header received
0811 15:12:35.822115 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:35.822118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:35.822171 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:35.822261 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:35.822615 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.823447 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.825215 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.825868 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.825980 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:35.826034 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:35.826258 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 15:12:35.826343 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260811 15:12:35.826486 29805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:35.821673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:47061 (local address 127.30.8.1:45715)
0811 15:12:35.821910 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:35.821915 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:35.821938 (+ 23us) server_negotiation.cc:407] Connection header received
0811 15:12:35.822115 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:35.822119 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:35.822173 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:35.822261 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:35.822617 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.823454 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.825219 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.825834 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.825956 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:35.826030 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:35.826258 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 15:12:35.826338 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260811 15:12:35.826537 29822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:35.821619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:39059 (local address 127.30.8.2:36937)
0811 15:12:35.821939 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:35.821950 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:35.822027 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:35.822278 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:35.822281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:35.822313 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:35.822501 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:35.822509 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.823577 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.823582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:35.825065 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:35.825078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.826003 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.826006 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:35.826124 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:35.826168 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 15:12:35.826230 (+ 62us) client_negotiation.cc:240] Negotiation successful
0811 15:12:35.826338 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"mutex_wait_us":130,"spinlock_wait_cycles":8448,"thread_start_us":99,"threads_started":1}
I20260811 15:12:35.826537 29821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:35.821555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:45715 (local address 127.30.8.2:47061)
0811 15:12:35.821866 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:35.821883 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:35.821999 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:35.822278 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:35.822281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:35.822306 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:35.822508 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:35.822514 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.823577 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.823582 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:35.825065 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:35.825078 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.825963 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.825967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:35.826070 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:35.826148 (+ 78us) client_negotiation.cc:769] Sending connection context
0811 15:12:35.826230 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 15:12:35.826338 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":232,"mutex_wait_us":212,"spinlock_wait_cycles":6272,"thread_start_us":117,"threads_started":1}
I20260811 15:12:35.827310 29705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e576771abe7f472895f636bc3321dcc8" candidate_uuid: "0b98a385304b40c28a9b87b291f378a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3707319f2e041e1aa4d3d1ff0fcd965" is_pre_election: true
I20260811 15:12:35.827332 29275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e576771abe7f472895f636bc3321dcc8" candidate_uuid: "0b98a385304b40c28a9b87b291f378a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb7d8eb084194232bfdb4f1e804c8bb9" is_pre_election: true
I20260811 15:12:35.827646 29705 raft_consensus.cc:2454] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b98a385304b40c28a9b87b291f378a8 in term 0.
I20260811 15:12:35.827652 29275 raft_consensus.cc:2454] T e576771abe7f472895f636bc3321dcc8 P eb7d8eb084194232bfdb4f1e804c8bb9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0b98a385304b40c28a9b87b291f378a8 in term 0.
I20260811 15:12:35.828159 29398 leader_election.cc:302] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b98a385304b40c28a9b87b291f378a8, f3707319f2e041e1aa4d3d1ff0fcd965; no voters:
I20260811 15:12:35.828452 29818 raft_consensus.cc:2781] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:12:35.828547 29818 raft_consensus.cc:482] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:12:35.828600 29818 raft_consensus.cc:3037] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:35.829716 29818 raft_consensus.cc:504] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b98a385304b40c28a9b87b291f378a8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44329 } } peers { permanent_uuid: "eb7d8eb084194232bfdb4f1e804c8bb9" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45715 } } peers { permanent_uuid: "f3707319f2e041e1aa4d3d1ff0fcd965" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 39059 } }
I20260811 15:12:35.830120 29818 leader_election.cc:288] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [CANDIDATE]: Term 1 election: Requested vote from peers eb7d8eb084194232bfdb4f1e804c8bb9 (127.30.8.1:45715), f3707319f2e041e1aa4d3d1ff0fcd965 (127.30.8.3:39059)
I20260811 15:12:35.830562 29705 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e576771abe7f472895f636bc3321dcc8" candidate_uuid: "0b98a385304b40c28a9b87b291f378a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f3707319f2e041e1aa4d3d1ff0fcd965"
I20260811 15:12:35.830705 29275 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e576771abe7f472895f636bc3321dcc8" candidate_uuid: "0b98a385304b40c28a9b87b291f378a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb7d8eb084194232bfdb4f1e804c8bb9"
I20260811 15:12:35.830705 29705 raft_consensus.cc:3037] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:35.830845 29275 raft_consensus.cc:3037] T e576771abe7f472895f636bc3321dcc8 P eb7d8eb084194232bfdb4f1e804c8bb9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:35.831933 29275 raft_consensus.cc:2454] T e576771abe7f472895f636bc3321dcc8 P eb7d8eb084194232bfdb4f1e804c8bb9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b98a385304b40c28a9b87b291f378a8 in term 1.
I20260811 15:12:35.831874 29705 raft_consensus.cc:2454] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0b98a385304b40c28a9b87b291f378a8 in term 1.
I20260811 15:12:35.832336 29396 leader_election.cc:302] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0b98a385304b40c28a9b87b291f378a8, eb7d8eb084194232bfdb4f1e804c8bb9; no voters:
I20260811 15:12:35.832577 29818 raft_consensus.cc:2781] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:35.832835 29818 raft_consensus.cc:686] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [term 1 LEADER]: Becoming Leader. State: Replica: 0b98a385304b40c28a9b87b291f378a8, State: Running, Role: LEADER
I20260811 15:12:35.833230 29818 consensus_queue.cc:237] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b98a385304b40c28a9b87b291f378a8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44329 } } peers { permanent_uuid: "eb7d8eb084194232bfdb4f1e804c8bb9" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45715 } } peers { permanent_uuid: "f3707319f2e041e1aa4d3d1ff0fcd965" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 39059 } }
I20260811 15:12:35.836613 29064 catalog_manager.cc:5495] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0b98a385304b40c28a9b87b291f378a8 (127.30.8.2). New cstate: current_term: 1 leader_uuid: "0b98a385304b40c28a9b87b291f378a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0b98a385304b40c28a9b87b291f378a8" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 44329 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb7d8eb084194232bfdb4f1e804c8bb9" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f3707319f2e041e1aa4d3d1ff0fcd965" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 39059 } health_report { overall_health: UNKNOWN } } }
I20260811 15:12:35.915421 29800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:35.912021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51152 (local address 127.30.8.2:44329)
0811 15:12:35.912143 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:35.912148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:35.912160 (+ 12us) server_negotiation.cc:407] Connection header received
0811 15:12:35.912237 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:35.912239 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:35.912296 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:35.912395 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:12:35.912582 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.913079 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.913944 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:35.914148 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:35.914217 (+ 69us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:35.914537 (+ 320us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:12:35.915017 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:12:35.915130 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:35.915261 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 15:12:35.915300 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 15:12:35.921763 29275 raft_consensus.cc:1261] T e576771abe7f472895f636bc3321dcc8 P eb7d8eb084194232bfdb4f1e804c8bb9 [term 1 FOLLOWER]: Refusing update from remote peer 0b98a385304b40c28a9b87b291f378a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:12:35.921763 29705 raft_consensus.cc:1261] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965 [term 1 FOLLOWER]: Refusing update from remote peer 0b98a385304b40c28a9b87b291f378a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:12:35.922528 29825 consensus_queue.cc:1035] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f3707319f2e041e1aa4d3d1ff0fcd965" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 39059 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:12:35.922726 29818 consensus_queue.cc:1035] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb7d8eb084194232bfdb4f1e804c8bb9" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:12:35.931988 29832 mvcc.cc:205] Tried to move back new op lower bound from 7317344894648053760 to 7317344894295805952. Current Snapshot: MvccSnapshot[applied={T|T < 7317344894648053760}]
I20260811 15:12:35.932524 29834 mvcc.cc:205] Tried to move back new op lower bound from 7317344894648053760 to 7317344894295805952. Current Snapshot: MvccSnapshot[applied={T|T < 7317344894648053760}]
I20260811 15:12:35.940295 29062 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:12:35.940466 29062 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48128:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:12:35.943009 29062 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 41941b9b1c3147bcb83580c4a15a0270: Sending DeleteTablet for 3 replicas of tablet e576771abe7f472895f636bc3321dcc8
I20260811 15:12:35.943976 29465 tablet_service.cc:1514] Processing DeleteTablet for tablet e576771abe7f472895f636bc3321dcc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:12:35 UTC) from {username='slave'} at 127.0.0.1:51144
I20260811 15:12:35.943986 29247 tablet_service.cc:1514] Processing DeleteTablet for tablet e576771abe7f472895f636bc3321dcc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:12:35 UTC) from {username='slave'} at 127.0.0.1:48466
I20260811 15:12:35.944171 29680 tablet_service.cc:1514] Processing DeleteTablet for tablet e576771abe7f472895f636bc3321dcc8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:12:35 UTC) from {username='slave'} at 127.0.0.1:33694
I20260811 15:12:35.944553 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 29156
I20260811 15:12:35.944680 29851 tablet_replica.cc:330] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965: stopping tablet replica
I20260811 15:12:35.945073 29851 raft_consensus.cc:2229] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:12:35.945493 29851 raft_consensus.cc:2258] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:12:35.945960 29846 tablet_replica.cc:330] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8: stopping tablet replica
I20260811 15:12:35.946300 29846 raft_consensus.cc:2229] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:12:35.946812 29846 raft_consensus.cc:2258] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:12:35.949395 29846 ts_tablet_manager.cc:1905] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:12:35.952852 29846 ts_tablet_manager.cc:1918] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:12:35.952996 29846 log.cc:1196] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/e576771abe7f472895f636bc3321dcc8
I20260811 15:12:35.953404 29846 ts_tablet_manager.cc:1939] T e576771abe7f472895f636bc3321dcc8 P 0b98a385304b40c28a9b87b291f378a8: Deleting consensus metadata
I20260811 15:12:35.954870 29039 catalog_manager.cc:4841] TS 0b98a385304b40c28a9b87b291f378a8 (127.30.8.2:44329): tablet e576771abe7f472895f636bc3321dcc8 (table test-table [id=65d69ab4137f4e4997b3e1c59b65c965]) successfully deleted
I20260811 15:12:35.956368 29851 ts_tablet_manager.cc:1905] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:12:35.959400 29851 ts_tablet_manager.cc:1918] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:12:35.959514 29851 log.cc:1196] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/e576771abe7f472895f636bc3321dcc8
I20260811 15:12:35.959839 29851 ts_tablet_manager.cc:1939] T e576771abe7f472895f636bc3321dcc8 P f3707319f2e041e1aa4d3d1ff0fcd965: Deleting consensus metadata
I20260811 15:12:35.961354 29043 catalog_manager.cc:4841] TS f3707319f2e041e1aa4d3d1ff0fcd965 (127.30.8.3:39059): tablet e576771abe7f472895f636bc3321dcc8 (table test-table [id=65d69ab4137f4e4997b3e1c59b65c965]) successfully deleted
W20260811 15:12:35.961944 29039 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.8.1:45715 (error 108)
I20260811 15:12:35.962172 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 29369
I20260811 15:12:35.972151 29797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:35.971730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:45715 (local address 127.0.0.1:48472)
0811 15:12:35.971962 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:35.972045 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.1:45715: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144}
W20260811 15:12:35.972422 29039 catalog_manager.cc:4584] TS eb7d8eb084194232bfdb4f1e804c8bb9 (127.30.8.1:45715): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e576771abe7f472895f636bc3321dcc8: Network error: Client connection negotiation failed: client connection to 127.30.8.1:45715: connect: Connection refused (error 111)
I20260811 15:12:35.984457 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 29580
I20260811 15:12:36.003136 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 29016
2026-08-11T15:12:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1413 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:12:36 dist-test-slave-1vmg krb5kdc[29858](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:12:36 dist-test-slave-1vmg krb5kdc[29858](info): set up 2 sockets
Aug 11 15:12:36 dist-test-slave-1vmg krb5kdc[29858](info): commencing operation
krb5kdc: starting...
W20260811 15:12:37.188391 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:12:37 dist-test-slave-1vmg krb5kdc[29858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:12:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:12:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:12:37.330214 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:38577
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:38577
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:37.446170 29881 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:37.446456 29881 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:37.446514 29881 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:37.450508 29881 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:12:37.450582 29881 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:37.450609 29881 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:37.450630 29881 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:12:37.450650 29881 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:12:37.456072 29881 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:38577
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:38577
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29881
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:37.457202 29881 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:37.458241 29881 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:37.466531 29890 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:37.466552 29889 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:37.466552 29894 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:37.466765 29881 server_base.cc:1034] running on GCE node
I20260811 15:12:37.467420 29881 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:37.468549 29881 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:37.469719 29881 hybrid_clock.cc:648] HybridClock initialized: now 1786461157469700 us; error 30 us; skew 500 ppm
Aug 11 15:12:37 dist-test-slave-1vmg krb5kdc[29858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461157, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:37.472433 29881 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:12:37.473367 29881 webserver.cc:460] Webserver started at http://127.30.8.62:46089/ using document root <none> and password file <none>
I20260811 15:12:37.473924 29881 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:37.473987 29881 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:37.474210 29881 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:37.476325 29881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "2aadd7580d8945888eed7ab092406257"
format_stamp: "Formatted at 2026-08-11 15:12:37 on dist-test-slave-1vmg"
server_key: "24fc63fb32ab883b5e908b77d4b5ac42"
server_key_iv: "76fad1efb5667451c9b2f8cbd080d8fe"
server_key_version: "encryptionkey@0"
I20260811 15:12:37.476874 29881 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "2aadd7580d8945888eed7ab092406257"
format_stamp: "Formatted at 2026-08-11 15:12:37 on dist-test-slave-1vmg"
server_key: "24fc63fb32ab883b5e908b77d4b5ac42"
server_key_iv: "76fad1efb5667451c9b2f8cbd080d8fe"
server_key_version: "encryptionkey@0"
I20260811 15:12:37.480645 29881 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:12:37.483034 29904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:37.484238 29881 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 15:12:37.484352 29881 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "2aadd7580d8945888eed7ab092406257"
format_stamp: "Formatted at 2026-08-11 15:12:37 on dist-test-slave-1vmg"
server_key: "24fc63fb32ab883b5e908b77d4b5ac42"
server_key_iv: "76fad1efb5667451c9b2f8cbd080d8fe"
server_key_version: "encryptionkey@0"
I20260811 15:12:37.484462 29881 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:37.505549 29881 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:37.508469 29881 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:37.508661 29881 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:37.515739 29881 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:38577
I20260811 15:12:37.515753 29987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:38577 every 8 connection(s)
I20260811 15:12:37.516731 29881 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:12:37.517967 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 29881
I20260811 15:12:37.518105 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:12:37.518432 30752 external_mini_cluster.cc:1428] Setting key 0ed649d11881a21174baa15dfe9f8668
I20260811 15:12:37.520147 29989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:37.526721 29989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257: Bootstrap starting.
Aug 11 15:12:37 dist-test-slave-1vmg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461157, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:12:37.529065 29989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:37.529795 29989 log.cc:824] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:37.531808 29989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257: No bootstrap required, opened a new log
I20260811 15:12:37.532272 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:37.519870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47942 (local address 127.30.8.62:38577)
0811 15:12:37.520277 (+ 407us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:37.520293 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:37.520328 (+ 35us) server_negotiation.cc:407] Connection header received
0811 15:12:37.520944 (+ 616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:37.520959 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:37.521280 (+ 321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:37.521551 (+ 271us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:37.522411 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:37.523453 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:37.524544 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:37.524990 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:37.525110 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:37.527897 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:37.527902 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:37.527916 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:37.527945 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:37.529544 (+ 1599us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:37.529951 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:37.529956 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:37.529961 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:37.530084 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:37.530339 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:37.530342 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:37.530343 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:37.530744 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:37.530974 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:37.531243 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 15:12:37.531531 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":119,"threads_started":1}
I20260811 15:12:37.535506 29989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aadd7580d8945888eed7ab092406257" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 38577 } }
I20260811 15:12:37.535738 29989 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:37.535799 29989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2aadd7580d8945888eed7ab092406257, State: Initialized, Role: FOLLOWER
I20260811 15:12:37.536198 29989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aadd7580d8945888eed7ab092406257" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 38577 } }
I20260811 15:12:37.536329 29989 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:12:37.536393 29989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:12:37.536494 29989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:37.537400 29989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aadd7580d8945888eed7ab092406257" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 38577 } }
I20260811 15:12:37.537732 29989 leader_election.cc:302] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2aadd7580d8945888eed7ab092406257; no voters:
I20260811 15:12:37.538056 29989 leader_election.cc:288] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:12:37.538200 29995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:37.538424 29995 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257 [term 1 LEADER]: Becoming Leader. State: Replica: 2aadd7580d8945888eed7ab092406257, State: Running, Role: LEADER
I20260811 15:12:37.538750 29995 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aadd7580d8945888eed7ab092406257" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 38577 } }
I20260811 15:12:37.539324 29989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:12:37.540392 29996 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2aadd7580d8945888eed7ab092406257. Latest consensus state: current_term: 1 leader_uuid: "2aadd7580d8945888eed7ab092406257" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aadd7580d8945888eed7ab092406257" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 38577 } } }
I20260811 15:12:37.540525 29996 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257 [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:37.540789 29997 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2aadd7580d8945888eed7ab092406257" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2aadd7580d8945888eed7ab092406257" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 38577 } } }
I20260811 15:12:37.540884 29997 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257 [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:37.541059 30006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:12:37.543910 30006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:12:37.549078 30006 catalog_manager.cc:1347] Generated new cluster ID: eb45b3fb368c476092b90c33b2166705
I20260811 15:12:37.549150 30006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:12:37.559484 30006 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:12:37.560801 30006 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:12:37.568051 30006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257: Generated new TSK 0
I20260811 15:12:37.568758 30006 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.8.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
Entry for principal kudu/127.30.8.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab.
I20260811 15:12:37.597890 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:38577
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:37.713497 30028 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:37.713792 30028 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:37.713838 30028 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:37.717938 30028 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:37.718019 30028 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:37.718097 30028 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:12:37.723583 30028 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.1.keytab
--principal=kudu/127.30.8.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:38577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30028
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:37.724762 30028 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:37.725806 30028 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:37.735155 30039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:37.735204 30028 server_base.cc:1034] running on GCE node
W20260811 15:12:37.735152 30040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:37.735147 30042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:37.735769 30028 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:37.736392 30028 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:37.737548 30028 hybrid_clock.cc:648] HybridClock initialized: now 1786461157737524 us; error 38 us; skew 500 ppm
Aug 11 15:12:37 dist-test-slave-1vmg krb5kdc[29858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461157, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:37.740425 30028 init.cc:377] Logged in from keytab as kudu/127.30.8.1@KRBTEST.COM (short username kudu)
I20260811 15:12:37.741444 30028 webserver.cc:460] Webserver started at http://127.30.8.1:35873/ using document root <none> and password file <none>
I20260811 15:12:37.742008 30028 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:37.742069 30028 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:37.742280 30028 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:37.744606 30028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "e35b39c5145647eca5b251df1ecabe86"
format_stamp: "Formatted at 2026-08-11 15:12:37 on dist-test-slave-1vmg"
server_key: "90d9e9132040da43630d82c1add6c686"
server_key_iv: "e795817282c6351bb63db5984dd3cbf8"
server_key_version: "encryptionkey@0"
I20260811 15:12:37.745178 30028 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "e35b39c5145647eca5b251df1ecabe86"
format_stamp: "Formatted at 2026-08-11 15:12:37 on dist-test-slave-1vmg"
server_key: "90d9e9132040da43630d82c1add6c686"
server_key_iv: "e795817282c6351bb63db5984dd3cbf8"
server_key_version: "encryptionkey@0"
I20260811 15:12:37.748939 30028 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:12:37.751319 30053 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:37.752482 30028 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:12:37.752594 30028 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "e35b39c5145647eca5b251df1ecabe86"
format_stamp: "Formatted at 2026-08-11 15:12:37 on dist-test-slave-1vmg"
server_key: "90d9e9132040da43630d82c1add6c686"
server_key_iv: "e795817282c6351bb63db5984dd3cbf8"
server_key_version: "encryptionkey@0"
I20260811 15:12:37.752701 30028 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:37.784665 30028 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:37.787631 30028 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:37.787822 30028 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:37.788434 30028 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:37.789309 30028 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:12:37.789359 30028 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:37.789412 30028 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:12:37.789441 30028 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:37.797899 30028 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:45223
I20260811 15:12:37.797923 30225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:45223 every 8 connection(s)
I20260811 15:12:37.798800 30028 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:12:37.805799 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 30028
I20260811 15:12:37.805939 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:12:37.806211 30752 external_mini_cluster.cc:1428] Setting key baf3c3390a6af0694927a8eb87fcecac
Aug 11 15:12:37 dist-test-slave-1vmg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461157, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.1@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:12:37.813192 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:37.800654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:39861 (local address 127.30.8.62:38577)
0811 15:12:37.800798 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:37.800802 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:37.801585 (+ 783us) server_negotiation.cc:407] Connection header received
0811 15:12:37.802862 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:37.802866 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:37.802958 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:37.803033 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:37.804218 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:37.804757 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:37.805898 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:37.806202 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:37.806296 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:37.809067 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:37.809071 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:37.809073 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:37.809096 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:37.810522 (+ 1426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:37.811047 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:37.811050 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:37.811052 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:37.811151 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:37.811520 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:37.811524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:37.811525 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:37.811775 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:37.811876 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:37.812753 (+ 877us) server_negotiation.cc:299] Negotiation successful
0811 15:12:37.812954 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 15:12:37.814064 30230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:37.801002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:38577 (local address 127.30.8.1:39861)
0811 15:12:37.801445 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:37.801485 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:37.802649 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:37.803228 (+ 579us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:37.803234 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:37.803593 (+ 359us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:37.804071 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:37.804083 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:37.804878 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:37.804881 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:37.805778 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:37.805788 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:37.806351 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:37.806355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:37.806518 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:37.807118 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:12:37.807150 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:37.808899 (+ 1749us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:37.810646 (+ 1747us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:37.810651 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:37.810664 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:37.810950 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:37.811232 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:37.811234 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:37.811235 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:37.811411 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:37.811910 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:37.811917 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:37.812405 (+ 488us) client_negotiation.cc:769] Sending connection context
0811 15:12:37.812630 (+ 225us) client_negotiation.cc:240] Negotiation successful
0811 15:12:37.812989 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":104,"threads_started":1}
I20260811 15:12:37.815436 30226 heartbeater.cc:347] Connected to a master server at 127.30.8.62:38577
I20260811 15:12:37.815687 30226 heartbeater.cc:464] Registering TS with master...
I20260811 15:12:37.816193 30226 heartbeater.cc:511] Master 127.30.8.62:38577 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.8.2@KRBTEST.COM; defaulting to no policy
I20260811 15:12:37.817780 29931 ts_manager.cc:194] Registered new tserver with Master: e35b39c5145647eca5b251df1ecabe86 (127.30.8.1:45223)
I20260811 15:12:37.819023 29931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.1@KRBTEST.COM'} at 127.30.8.1:39861
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.2@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
Entry for principal kudu/127.30.8.2 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab.
I20260811 15:12:37.835307 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:38577
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:37.949313 30236 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:37.949580 30236 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:37.949626 30236 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:37.953665 30236 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:37.953737 30236 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:37.953810 30236 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:12:37.959313 30236 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.2.keytab
--principal=kudu/127.30.8.2
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:38577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30236
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:37.960464 30236 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:37.961500 30236 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:37.971073 30250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:37.971130 30251 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:37.971078 30253 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:37.971138 30236 server_base.cc:1034] running on GCE node
I20260811 15:12:37.971864 30236 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:37.972532 30236 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:37.973698 30236 hybrid_clock.cc:648] HybridClock initialized: now 1786461157973670 us; error 39 us; skew 500 ppm
Aug 11 15:12:37 dist-test-slave-1vmg krb5kdc[29858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461157, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:37.976610 30236 init.cc:377] Logged in from keytab as kudu/127.30.8.2@KRBTEST.COM (short username kudu)
I20260811 15:12:37.977530 30236 webserver.cc:460] Webserver started at http://127.30.8.2:43897/ using document root <none> and password file <none>
I20260811 15:12:37.978083 30236 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:37.978148 30236 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:37.978341 30236 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:37.980513 30236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "80a75cbce27e443b8ce6a04760de2254"
format_stamp: "Formatted at 2026-08-11 15:12:37 on dist-test-slave-1vmg"
server_key: "129f898954d8637a197c1047dfba5daa"
server_key_iv: "b45259d95137edad2a7cb07548741edc"
server_key_version: "encryptionkey@0"
I20260811 15:12:37.981078 30236 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "80a75cbce27e443b8ce6a04760de2254"
format_stamp: "Formatted at 2026-08-11 15:12:37 on dist-test-slave-1vmg"
server_key: "129f898954d8637a197c1047dfba5daa"
server_key_iv: "b45259d95137edad2a7cb07548741edc"
server_key_version: "encryptionkey@0"
I20260811 15:12:37.985211 30236 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:12:37.987730 30265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:37.988988 30236 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:12:37.989107 30236 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "80a75cbce27e443b8ce6a04760de2254"
format_stamp: "Formatted at 2026-08-11 15:12:37 on dist-test-slave-1vmg"
server_key: "129f898954d8637a197c1047dfba5daa"
server_key_iv: "b45259d95137edad2a7cb07548741edc"
server_key_version: "encryptionkey@0"
I20260811 15:12:37.989229 30236 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:38.001511 30236 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:38.004890 30236 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:38.005086 30236 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:38.005688 30236 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:38.006575 30236 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:12:38.006629 30236 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:38.006683 30236 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:12:38.006707 30236 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:38.016290 30236 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:42977
I20260811 15:12:38.016307 30441 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:42977 every 8 connection(s)
I20260811 15:12:38.017261 30236 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:12:38.022287 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 30236
I20260811 15:12:38.022436 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:12:38.022801 30752 external_mini_cluster.cc:1428] Setting key 38b5a3a37ef2495033563a6df5907780
Aug 11 15:12:38 dist-test-slave-1vmg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461157, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.2@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:12:38.031167 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:38.019163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:54625 (local address 127.30.8.62:38577)
0811 15:12:38.019336 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:38.019342 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:38.020414 (+ 1072us) server_negotiation.cc:407] Connection header received
0811 15:12:38.021233 (+ 819us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:38.021235 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:38.021286 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:38.021368 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:38.022531 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.023307 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.024521 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.024835 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.024955 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:38.027337 (+ 2382us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:38.027342 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:38.027345 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:38.027377 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:38.028839 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:38.029425 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:38.029427 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:38.029429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:38.029516 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:38.029899 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:38.029902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:38.029903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:38.030098 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:38.030181 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:38.030797 (+ 616us) server_negotiation.cc:299] Negotiation successful
0811 15:12:38.030967 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 15:12:38.031915 30444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:38.019594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:38577 (local address 127.30.8.2:54625)
0811 15:12:38.020269 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:38.020310 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:38.021052 (+ 742us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:38.021509 (+ 457us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:38.021515 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:38.021869 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:38.022364 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:38.022377 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.023441 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.023444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:38.024402 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:38.024411 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.024972 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.024974 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:38.025110 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:38.025681 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:12:38.025697 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:38.027164 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:38.028960 (+ 1796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:38.028967 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:38.028982 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:38.029332 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:38.029602 (+ 270us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:38.029605 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:38.029607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:38.029807 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:38.030201 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:38.030208 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:38.030505 (+ 297us) client_negotiation.cc:769] Sending connection context
0811 15:12:38.030715 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 15:12:38.031039 (+ 324us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":128,"threads_started":1}
I20260811 15:12:38.033313 30442 heartbeater.cc:347] Connected to a master server at 127.30.8.62:38577
I20260811 15:12:38.033548 30442 heartbeater.cc:464] Registering TS with master...
I20260811 15:12:38.034039 30442 heartbeater.cc:511] Master 127.30.8.62:38577 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.8.3@KRBTEST.COM; defaulting to no policy
I20260811 15:12:38.035197 29931 ts_manager.cc:194] Registered new tserver with Master: 80a75cbce27e443b8ce6a04760de2254 (127.30.8.2:42977)
I20260811 15:12:38.035802 29931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.2@KRBTEST.COM'} at 127.30.8.2:54625
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.3@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
Entry for principal kudu/127.30.8.3 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab.
I20260811 15:12:38.052018 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:38577
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:44841
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:38.167630 30448 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:38.167898 30448 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:38.167958 30448 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:38.172046 30448 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:38.172115 30448 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:38.172202 30448 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:12:38.177610 30448 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:44841
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.3.keytab
--principal=kudu/127.30.8.3
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:38577
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30448
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:38.178818 30448 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:38.179930 30448 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:38.190099 30456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:38.190099 30454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:38.190112 30461 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:38.190440 30448 server_base.cc:1034] running on GCE node
I20260811 15:12:38.190881 30448 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:38.191589 30448 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:38.192739 30448 hybrid_clock.cc:648] HybridClock initialized: now 1786461158192711 us; error 46 us; skew 500 ppm
Aug 11 15:12:38 dist-test-slave-1vmg krb5kdc[29858](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461158, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:38.195542 30448 init.cc:377] Logged in from keytab as kudu/127.30.8.3@KRBTEST.COM (short username kudu)
I20260811 15:12:38.197034 30448 webserver.cc:460] Webserver started at http://127.30.8.3:35399/ using document root <none> and password file <none>
I20260811 15:12:38.197840 30448 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:38.197926 30448 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:38.198177 30448 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:38.200855 30448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "51ace2bbef6647b49fbb809794d1b19c"
format_stamp: "Formatted at 2026-08-11 15:12:38 on dist-test-slave-1vmg"
server_key: "1557528dea06b2cd52bce0f02f9475a3"
server_key_iv: "9b2e2e962afd3c0d09151d2f75cd88d6"
server_key_version: "encryptionkey@0"
I20260811 15:12:38.201522 30448 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "51ace2bbef6647b49fbb809794d1b19c"
format_stamp: "Formatted at 2026-08-11 15:12:38 on dist-test-slave-1vmg"
server_key: "1557528dea06b2cd52bce0f02f9475a3"
server_key_iv: "9b2e2e962afd3c0d09151d2f75cd88d6"
server_key_version: "encryptionkey@0"
I20260811 15:12:38.206271 30448 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 15:12:38.209216 30473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:38.210479 30448 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 15:12:38.210601 30448 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "51ace2bbef6647b49fbb809794d1b19c"
format_stamp: "Formatted at 2026-08-11 15:12:38 on dist-test-slave-1vmg"
server_key: "1557528dea06b2cd52bce0f02f9475a3"
server_key_iv: "9b2e2e962afd3c0d09151d2f75cd88d6"
server_key_version: "encryptionkey@0"
I20260811 15:12:38.210728 30448 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:38.229096 30448 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:38.232107 30448 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:38.232311 30448 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:38.232937 30448 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:38.233829 30448 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:12:38.233878 30448 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:38.233933 30448 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:12:38.233963 30448 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:38.243331 30448 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:40441
I20260811 15:12:38.243360 30646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:40441 every 8 connection(s)
I20260811 15:12:38.244289 30448 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:12:38.249624 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 30448
I20260811 15:12:38.249766 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:12:38.250034 30752 external_mini_cluster.cc:1428] Setting key 3f7d78a7c02c98e77896cada05be5f89
Aug 11 15:12:38 dist-test-slave-1vmg krb5kdc[29858](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461158, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.3@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:12:38.257735 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:38.246121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:60899 (local address 127.30.8.62:38577)
0811 15:12:38.246260 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:38.246264 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:38.247418 (+ 1154us) server_negotiation.cc:407] Connection header received
0811 15:12:38.248359 (+ 941us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:38.248363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:38.248423 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:38.248497 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:38.249732 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.250322 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.251771 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.252000 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.252112 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:38.254614 (+ 2502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:38.254617 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:38.254619 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:38.254645 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:38.255852 (+ 1207us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:38.256369 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:38.256372 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:38.256373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:38.256455 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:38.256733 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:38.256735 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:38.256736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:38.256911 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:38.256985 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:38.257484 (+ 499us) server_negotiation.cc:299] Negotiation successful
0811 15:12:38.257607 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 15:12:38.258597 30649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:38.246499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:38577 (local address 127.30.8.3:60899)
0811 15:12:38.247264 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:38.247320 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:38.248138 (+ 818us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:38.248693 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:38.248700 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:38.249062 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:38.249571 (+ 509us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:38.249586 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.250448 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.250452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:38.251662 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:38.251675 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.252169 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.252174 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:38.252374 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:38.252981 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:12:38.252998 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:38.254457 (+ 1459us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:38.255962 (+ 1505us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:38.255968 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:38.255983 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:38.256292 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:38.256527 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:12:38.256529 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:12:38.256530 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:12:38.256667 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:12:38.256983 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:38.256988 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:38.257259 (+ 271us) client_negotiation.cc:769] Sending connection context
0811 15:12:38.257441 (+ 182us) client_negotiation.cc:240] Negotiation successful
0811 15:12:38.257736 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":110,"threads_started":1}
I20260811 15:12:38.259765 30647 heartbeater.cc:347] Connected to a master server at 127.30.8.62:38577
I20260811 15:12:38.260003 30647 heartbeater.cc:464] Registering TS with master...
I20260811 15:12:38.260509 30647 heartbeater.cc:511] Master 127.30.8.62:38577 requested a full tablet report, sending...
I20260811 15:12:38.261569 29931 ts_manager.cc:194] Registered new tserver with Master: 51ace2bbef6647b49fbb809794d1b19c (127.30.8.3:40441)
I20260811 15:12:38.262140 29931 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.8.3@KRBTEST.COM'} at 127.30.8.3:60899
I20260811 15:12:38.264423 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 15:12:38.273058 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:38.266313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47952 (local address 127.30.8.62:38577)
0811 15:12:38.266458 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:38.266463 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:38.266566 (+ 103us) server_negotiation.cc:407] Connection header received
0811 15:12:38.266670 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:38.266672 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:38.266710 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:38.266771 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:38.267497 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.267997 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.268772 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.269061 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.269143 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:38.270452 (+ 1309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:38.270454 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:38.270456 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:38.270472 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:38.271658 (+ 1186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:38.272015 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:38.272018 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:38.272019 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:38.272100 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:38.272300 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:38.272302 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:38.272303 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:38.272502 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:38.272606 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:38.272772 (+ 166us) server_negotiation.cc:299] Negotiation successful
0811 15:12:38.272876 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 15:12:38.276161 29931 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47952:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:12:38.279194 29931 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:12:38.295408 30669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:38.288428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44032 (local address 127.30.8.2:42977)
0811 15:12:38.288859 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:38.288873 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:38.289596 (+ 723us) server_negotiation.cc:407] Connection header received
0811 15:12:38.289673 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:38.289677 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:38.289872 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:38.290027 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:38.290395 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.291585 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.293904 (+ 2319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.294538 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.294659 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:38.294836 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:38.295096 (+ 260us) server_negotiation.cc:299] Negotiation successful
0811 15:12:38.295215 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":108,"threads_started":1}
I20260811 15:12:38.295651 30672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:38.289205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59168 (local address 127.30.8.1:45223)
0811 15:12:38.289914 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:38.289927 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:38.289944 (+ 17us) server_negotiation.cc:407] Connection header received
0811 15:12:38.290002 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:38.290005 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:38.290128 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:38.290263 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:38.290499 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.291395 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.293640 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.294627 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.294754 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:38.295007 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:38.295320 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 15:12:38.295454 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":622,"thread_start_us":80,"threads_started":1}
I20260811 15:12:38.295639 30671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:38.288278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:40441 (local address 127.0.0.1:40726)
0811 15:12:38.289365 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:38.289378 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:38.289451 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:38.290823 (+ 1372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:38.290827 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:38.290847 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:38.290971 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:38.290977 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.292341 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.292345 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:38.293708 (+ 1363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:38.293723 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.295074 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.295078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:38.295233 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:38.295263 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 15:12:38.295354 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 15:12:38.295458 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1027,"mutex_wait_us":86,"thread_start_us":102,"threads_started":1}
I20260811 15:12:38.296119 30667 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:38.289020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:45223 (local address 127.0.0.1:59168)
0811 15:12:38.289615 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:38.289628 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:38.289710 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:38.290261 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:38.290264 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:38.290282 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:38.290389 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:38.290397 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.291765 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.291771 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:38.293485 (+ 1714us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:38.293500 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.294962 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.294966 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:38.295076 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:38.295171 (+ 95us) client_negotiation.cc:769] Sending connection context
0811 15:12:38.295261 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 15:12:38.295431 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":534,"mutex_wait_us":143,"thread_start_us":118,"threads_started":1}
I20260811 15:12:38.296344 30668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:38.288234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:42977 (local address 127.0.0.1:44032)
0811 15:12:38.289239 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:38.289294 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:38.289444 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:38.290040 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:38.290046 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:38.290070 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:38.290234 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:38.290242 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.291767 (+ 1525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.291944 (+ 177us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:38.293751 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:38.293759 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.294676 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.294681 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:38.294798 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:38.294842 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 15:12:38.295950 (+ 1108us) client_negotiation.cc:240] Negotiation successful
0811 15:12:38.296062 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"mutex_wait_us":114,"thread_start_us":108,"threads_started":1}
I20260811 15:12:38.296661 30670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:38.288950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40726 (local address 127.30.8.3:40441)
0811 15:12:38.290363 (+ 1413us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:38.290377 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:38.290399 (+ 22us) server_negotiation.cc:407] Connection header received
0811 15:12:38.290489 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:38.290494 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:38.290669 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:38.290829 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:38.291103 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.292129 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.293889 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.294713 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.294811 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:38.294997 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:38.296318 (+ 1321us) server_negotiation.cc:299] Negotiation successful
0811 15:12:38.296472 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1309,"thread_start_us":180,"threads_started":1}
I20260811 15:12:38.299546 30548 tablet_service.cc:1467] Processing CreateTablet for tablet 5c9582ad78374c7291fff6616db92fd0 (DEFAULT_TABLE table=test-table [id=45dc7635828242f8aabe68024bd26da8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:12:38.299543 30334 tablet_service.cc:1467] Processing CreateTablet for tablet 5c9582ad78374c7291fff6616db92fd0 (DEFAULT_TABLE table=test-table [id=45dc7635828242f8aabe68024bd26da8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:12:38.299501 30128 tablet_service.cc:1467] Processing CreateTablet for tablet 5c9582ad78374c7291fff6616db92fd0 (DEFAULT_TABLE table=test-table [id=45dc7635828242f8aabe68024bd26da8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:12:38.300750 30548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c9582ad78374c7291fff6616db92fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:38.300750 30334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c9582ad78374c7291fff6616db92fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:38.300752 30128 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5c9582ad78374c7291fff6616db92fd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:38.307288 30673 tablet_bootstrap.cc:492] T 5c9582ad78374c7291fff6616db92fd0 P e35b39c5145647eca5b251df1ecabe86: Bootstrap starting.
I20260811 15:12:38.308671 30674 tablet_bootstrap.cc:492] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254: Bootstrap starting.
I20260811 15:12:38.308820 30675 tablet_bootstrap.cc:492] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c: Bootstrap starting.
I20260811 15:12:38.309610 30673 tablet_bootstrap.cc:654] T 5c9582ad78374c7291fff6616db92fd0 P e35b39c5145647eca5b251df1ecabe86: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:38.310732 30673 log.cc:824] T 5c9582ad78374c7291fff6616db92fd0 P e35b39c5145647eca5b251df1ecabe86: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:38.310794 30675 tablet_bootstrap.cc:654] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:38.311237 30674 tablet_bootstrap.cc:654] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:38.312052 30674 log.cc:824] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:38.312052 30675 log.cc:824] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:38.313073 30673 tablet_bootstrap.cc:492] T 5c9582ad78374c7291fff6616db92fd0 P e35b39c5145647eca5b251df1ecabe86: No bootstrap required, opened a new log
I20260811 15:12:38.313293 30673 ts_tablet_manager.cc:1397] T 5c9582ad78374c7291fff6616db92fd0 P e35b39c5145647eca5b251df1ecabe86: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 15:12:38.314108 30675 tablet_bootstrap.cc:492] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c: No bootstrap required, opened a new log
I20260811 15:12:38.314291 30675 ts_tablet_manager.cc:1397] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 15:12:38.314365 30674 tablet_bootstrap.cc:492] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254: No bootstrap required, opened a new log
I20260811 15:12:38.314577 30674 ts_tablet_manager.cc:1397] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.005s
I20260811 15:12:38.316942 30673 raft_consensus.cc:348] T 5c9582ad78374c7291fff6616db92fd0 P e35b39c5145647eca5b251df1ecabe86 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a75cbce27e443b8ce6a04760de2254" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42977 } } peers { permanent_uuid: "51ace2bbef6647b49fbb809794d1b19c" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 40441 } } peers { permanent_uuid: "e35b39c5145647eca5b251df1ecabe86" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45223 } }
I20260811 15:12:38.317189 30673 raft_consensus.cc:374] T 5c9582ad78374c7291fff6616db92fd0 P e35b39c5145647eca5b251df1ecabe86 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:38.317245 30673 raft_consensus.cc:729] T 5c9582ad78374c7291fff6616db92fd0 P e35b39c5145647eca5b251df1ecabe86 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e35b39c5145647eca5b251df1ecabe86, State: Initialized, Role: FOLLOWER
I20260811 15:12:38.317718 30673 consensus_queue.cc:260] T 5c9582ad78374c7291fff6616db92fd0 P e35b39c5145647eca5b251df1ecabe86 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a75cbce27e443b8ce6a04760de2254" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42977 } } peers { permanent_uuid: "51ace2bbef6647b49fbb809794d1b19c" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 40441 } } peers { permanent_uuid: "e35b39c5145647eca5b251df1ecabe86" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45223 } }
I20260811 15:12:38.318243 30675 raft_consensus.cc:348] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a75cbce27e443b8ce6a04760de2254" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42977 } } peers { permanent_uuid: "51ace2bbef6647b49fbb809794d1b19c" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 40441 } } peers { permanent_uuid: "e35b39c5145647eca5b251df1ecabe86" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45223 } }
I20260811 15:12:38.318248 30674 raft_consensus.cc:348] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a75cbce27e443b8ce6a04760de2254" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42977 } } peers { permanent_uuid: "51ace2bbef6647b49fbb809794d1b19c" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 40441 } } peers { permanent_uuid: "e35b39c5145647eca5b251df1ecabe86" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45223 } }
I20260811 15:12:38.318500 30675 raft_consensus.cc:374] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:38.318500 30674 raft_consensus.cc:374] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:38.318549 30674 raft_consensus.cc:729] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80a75cbce27e443b8ce6a04760de2254, State: Initialized, Role: FOLLOWER
I20260811 15:12:38.318549 30675 raft_consensus.cc:729] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51ace2bbef6647b49fbb809794d1b19c, State: Initialized, Role: FOLLOWER
I20260811 15:12:38.318747 30673 ts_tablet_manager.cc:1428] T 5c9582ad78374c7291fff6616db92fd0 P e35b39c5145647eca5b251df1ecabe86: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 15:12:38.319020 30674 consensus_queue.cc:260] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a75cbce27e443b8ce6a04760de2254" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42977 } } peers { permanent_uuid: "51ace2bbef6647b49fbb809794d1b19c" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 40441 } } peers { permanent_uuid: "e35b39c5145647eca5b251df1ecabe86" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45223 } }
I20260811 15:12:38.319034 30675 consensus_queue.cc:260] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a75cbce27e443b8ce6a04760de2254" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42977 } } peers { permanent_uuid: "51ace2bbef6647b49fbb809794d1b19c" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 40441 } } peers { permanent_uuid: "e35b39c5145647eca5b251df1ecabe86" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45223 } }
I20260811 15:12:38.319168 30226 heartbeater.cc:503] Master 127.30.8.62:38577 was elected leader, sending a full tablet report...
I20260811 15:12:38.319700 30442 heartbeater.cc:503] Master 127.30.8.62:38577 was elected leader, sending a full tablet report...
I20260811 15:12:38.319993 30647 heartbeater.cc:503] Master 127.30.8.62:38577 was elected leader, sending a full tablet report...
I20260811 15:12:38.320430 30675 ts_tablet_manager.cc:1428] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 15:12:38.320928 30674 ts_tablet_manager.cc:1428] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 15:12:38.365515 30682 raft_consensus.cc:482] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:12:38.365895 30682 raft_consensus.cc:504] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a75cbce27e443b8ce6a04760de2254" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42977 } } peers { permanent_uuid: "51ace2bbef6647b49fbb809794d1b19c" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 40441 } } peers { permanent_uuid: "e35b39c5145647eca5b251df1ecabe86" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45223 } }
I20260811 15:12:38.368284 30682 leader_election.cc:288] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 80a75cbce27e443b8ce6a04760de2254 (127.30.8.2:42977), e35b39c5145647eca5b251df1ecabe86 (127.30.8.1:45223)
I20260811 15:12:38.373939 30672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:38.369049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:49397 (local address 127.30.8.1:45223)
0811 15:12:38.369194 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:38.369199 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:38.369326 (+ 127us) server_negotiation.cc:407] Connection header received
0811 15:12:38.369475 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:38.369477 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:38.369527 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:38.369615 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:38.369898 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.370723 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.372375 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.373060 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.373178 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:38.373255 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:38.373686 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 15:12:38.373790 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 15:12:38.373939 30669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:38.368473 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:46363 (local address 127.30.8.2:42977)
0811 15:12:38.369014 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:38.369021 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:38.369042 (+ 21us) server_negotiation.cc:407] Connection header received
0811 15:12:38.369124 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:38.369128 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:38.369189 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:38.369296 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:38.369599 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.370508 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.372378 (+ 1870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.373069 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.373178 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:38.373255 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:38.373688 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 15:12:38.373796 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361}
I20260811 15:12:38.373929 30649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:38.368364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:42977 (local address 127.30.8.3:46363)
0811 15:12:38.368623 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:38.368641 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:38.368736 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:38.369323 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:38.369326 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:38.369342 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:38.369481 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:38.369488 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.370663 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.370666 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:38.372132 (+ 1466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:38.372143 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.373211 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.373217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:38.373452 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:38.373517 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 15:12:38.373613 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 15:12:38.373713 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":125}
I20260811 15:12:38.373942 30683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:38.368687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:45223 (local address 127.30.8.3:49397)
0811 15:12:38.369210 (+ 523us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:38.369223 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:38.369352 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:38.369644 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:38.369647 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:38.369665 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:38.369772 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:38.369776 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.370857 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.370860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:38.372131 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:38.372143 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.373211 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.373217 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:38.373455 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:38.373520 (+ 65us) client_negotiation.cc:769] Sending connection context
0811 15:12:38.373613 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 15:12:38.373713 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"mutex_wait_us":105,"thread_start_us":199,"threads_started":1}
I20260811 15:12:38.374848 30158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c9582ad78374c7291fff6616db92fd0" candidate_uuid: "51ace2bbef6647b49fbb809794d1b19c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e35b39c5145647eca5b251df1ecabe86" is_pre_election: true
I20260811 15:12:38.374845 30365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c9582ad78374c7291fff6616db92fd0" candidate_uuid: "51ace2bbef6647b49fbb809794d1b19c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80a75cbce27e443b8ce6a04760de2254" is_pre_election: true
I20260811 15:12:38.375241 30158 raft_consensus.cc:2454] T 5c9582ad78374c7291fff6616db92fd0 P e35b39c5145647eca5b251df1ecabe86 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51ace2bbef6647b49fbb809794d1b19c in term 0.
I20260811 15:12:38.375238 30365 raft_consensus.cc:2454] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 51ace2bbef6647b49fbb809794d1b19c in term 0.
I20260811 15:12:38.375778 30475 leader_election.cc:302] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51ace2bbef6647b49fbb809794d1b19c, e35b39c5145647eca5b251df1ecabe86; no voters:
I20260811 15:12:38.376087 30682 raft_consensus.cc:2781] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:12:38.376191 30682 raft_consensus.cc:482] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:12:38.376235 30682 raft_consensus.cc:3037] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:38.377378 30682 raft_consensus.cc:504] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a75cbce27e443b8ce6a04760de2254" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42977 } } peers { permanent_uuid: "51ace2bbef6647b49fbb809794d1b19c" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 40441 } } peers { permanent_uuid: "e35b39c5145647eca5b251df1ecabe86" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45223 } }
I20260811 15:12:38.377749 30682 leader_election.cc:288] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [CANDIDATE]: Term 1 election: Requested vote from peers 80a75cbce27e443b8ce6a04760de2254 (127.30.8.2:42977), e35b39c5145647eca5b251df1ecabe86 (127.30.8.1:45223)
I20260811 15:12:38.378225 30158 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c9582ad78374c7291fff6616db92fd0" candidate_uuid: "51ace2bbef6647b49fbb809794d1b19c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e35b39c5145647eca5b251df1ecabe86"
I20260811 15:12:38.378216 30365 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5c9582ad78374c7291fff6616db92fd0" candidate_uuid: "51ace2bbef6647b49fbb809794d1b19c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "80a75cbce27e443b8ce6a04760de2254"
I20260811 15:12:38.378371 30365 raft_consensus.cc:3037] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:38.378371 30158 raft_consensus.cc:3037] T 5c9582ad78374c7291fff6616db92fd0 P e35b39c5145647eca5b251df1ecabe86 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:38.379787 30365 raft_consensus.cc:2454] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51ace2bbef6647b49fbb809794d1b19c in term 1.
I20260811 15:12:38.379787 30158 raft_consensus.cc:2454] T 5c9582ad78374c7291fff6616db92fd0 P e35b39c5145647eca5b251df1ecabe86 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 51ace2bbef6647b49fbb809794d1b19c in term 1.
I20260811 15:12:38.380141 30475 leader_election.cc:302] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 51ace2bbef6647b49fbb809794d1b19c, e35b39c5145647eca5b251df1ecabe86; no voters:
I20260811 15:12:38.380331 30682 raft_consensus.cc:2781] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:38.380602 30682 raft_consensus.cc:686] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [term 1 LEADER]: Becoming Leader. State: Replica: 51ace2bbef6647b49fbb809794d1b19c, State: Running, Role: LEADER
I20260811 15:12:38.381013 30682 consensus_queue.cc:237] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a75cbce27e443b8ce6a04760de2254" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42977 } } peers { permanent_uuid: "51ace2bbef6647b49fbb809794d1b19c" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 40441 } } peers { permanent_uuid: "e35b39c5145647eca5b251df1ecabe86" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45223 } }
I20260811 15:12:38.384629 29927 catalog_manager.cc:5495] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c reported cstate change: term changed from 0 to 1, leader changed from <none> to 51ace2bbef6647b49fbb809794d1b19c (127.30.8.3). New cstate: current_term: 1 leader_uuid: "51ace2bbef6647b49fbb809794d1b19c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "80a75cbce27e443b8ce6a04760de2254" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42977 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "51ace2bbef6647b49fbb809794d1b19c" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 40441 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e35b39c5145647eca5b251df1ecabe86" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45223 } health_report { overall_health: UNKNOWN } } }
I20260811 15:12:38.396492 30670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:38.393028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40736 (local address 127.30.8.3:40441)
0811 15:12:38.393160 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:38.393164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:38.393178 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:12:38.393278 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:38.393280 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:38.393354 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:38.393440 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:12:38.393680 (+ 240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.394222 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.395129 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.395331 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.395405 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:38.395621 (+ 216us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:12:38.396071 (+ 450us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:12:38.396167 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:38.396293 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 15:12:38.396352 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 15:12:38.402966 30365 raft_consensus.cc:1261] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254 [term 1 FOLLOWER]: Refusing update from remote peer 51ace2bbef6647b49fbb809794d1b19c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:12:38.402966 30158 raft_consensus.cc:1261] T 5c9582ad78374c7291fff6616db92fd0 P e35b39c5145647eca5b251df1ecabe86 [term 1 FOLLOWER]: Refusing update from remote peer 51ace2bbef6647b49fbb809794d1b19c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:12:38.403790 30684 consensus_queue.cc:1035] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [LEADER]: Connected to new peer: Peer: permanent_uuid: "80a75cbce27e443b8ce6a04760de2254" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 42977 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:12:38.404080 30682 consensus_queue.cc:1035] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [LEADER]: Connected to new peer: Peer: permanent_uuid: "e35b39c5145647eca5b251df1ecabe86" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 45223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:12:38.423807 29927 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:12:38.424024 29927 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47952:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:12:38.426584 29927 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 2aadd7580d8945888eed7ab092406257: Sending DeleteTablet for 3 replicas of tablet 5c9582ad78374c7291fff6616db92fd0
I20260811 15:12:38.427536 30548 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c9582ad78374c7291fff6616db92fd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:12:38 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:40726
I20260811 15:12:38.427551 30334 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c9582ad78374c7291fff6616db92fd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:12:38 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:44032
I20260811 15:12:38.428160 30706 tablet_replica.cc:330] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254: stopping tablet replica
I20260811 15:12:38.428356 30128 tablet_service.cc:1514] Processing DeleteTablet for tablet 5c9582ad78374c7291fff6616db92fd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:12:38 UTC) from {username='kudu', principal='kudu/127.30.8.62@KRBTEST.COM'} at 127.0.0.1:59168
I20260811 15:12:38.428674 30706 raft_consensus.cc:2229] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:12:38.428896 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 30028
I20260811 15:12:38.429061 30706 raft_consensus.cc:2258] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:12:38.436297 30706 ts_tablet_manager.cc:1905] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:12:38.439314 30705 tablet_replica.cc:330] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c: stopping tablet replica
I20260811 15:12:38.439723 30705 raft_consensus.cc:2229] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:12:38.440655 30706 ts_tablet_manager.cc:1918] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:12:38.440768 30706 log.cc:1196] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/5c9582ad78374c7291fff6616db92fd0
I20260811 15:12:38.441135 30706 ts_tablet_manager.cc:1939] T 5c9582ad78374c7291fff6616db92fd0 P 80a75cbce27e443b8ce6a04760de2254: Deleting consensus metadata
I20260811 15:12:38.443254 30705 raft_consensus.cc:2258] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:12:38.443269 29907 catalog_manager.cc:4841] TS 80a75cbce27e443b8ce6a04760de2254 (127.30.8.2:42977): tablet 5c9582ad78374c7291fff6616db92fd0 (table test-table [id=45dc7635828242f8aabe68024bd26da8]) successfully deleted
I20260811 15:12:38.445290 30705 ts_tablet_manager.cc:1905] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 15:12:38.447343 29907 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.8.1:45223 (error 108)
I20260811 15:12:38.448671 30671 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:38.448339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:45223 (local address 127.0.0.1:59182)
0811 15:12:38.448487 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:38.448567 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.1:45223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 15:12:38.448758 30705 ts_tablet_manager.cc:1918] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:12:38.448840 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 30236
I20260811 15:12:38.448882 30705 log.cc:1196] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/5c9582ad78374c7291fff6616db92fd0
I20260811 15:12:38.449273 30705 ts_tablet_manager.cc:1939] T 5c9582ad78374c7291fff6616db92fd0 P 51ace2bbef6647b49fbb809794d1b19c: Deleting consensus metadata
W20260811 15:12:38.452745 29907 catalog_manager.cc:4584] TS e35b39c5145647eca5b251df1ecabe86 (127.30.8.1:45223): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5c9582ad78374c7291fff6616db92fd0: Network error: Client connection negotiation failed: client connection to 127.30.8.1:45223: connect: Connection refused (error 111)
I20260811 15:12:38.459057 29909 catalog_manager.cc:4841] TS 51ace2bbef6647b49fbb809794d1b19c (127.30.8.3:40441): tablet 5c9582ad78374c7291fff6616db92fd0 (table test-table [id=45dc7635828242f8aabe68024bd26da8]) successfully deleted
I20260811 15:12:38.467770 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 30448
I20260811 15:12:38.493733 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 29881
2026-08-11T15:12:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2483 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T15:12:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:12:38Z Disabled control of system clock
I20260811 15:12:38.536818 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:37125
--webserver_interface=127.30.8.62
--webserver_port=0
--builtin_ntp_servers=127.30.8.20:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:37125
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 15:12:38.656127 30713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:38.656433 30713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:38.656473 30713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:38.660490 30713 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:12:38.660563 30713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:38.660580 30713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:38.660604 30713 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:12:38.660629 30713 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:12:38.665598 30713 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:37125
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:37125
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:38.666756 30713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:38.668205 30713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:38.676842 30723 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:38.676846 30725 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:38.676846 30722 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:38.677150 30713 server_base.cc:1034] running on GCE node
I20260811 15:12:38.677788 30713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:38.678977 30713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:38.680156 30713 hybrid_clock.cc:648] HybridClock initialized: now 1786461158680135 us; error 35 us; skew 500 ppm
I20260811 15:12:38.686353 30713 webserver.cc:460] Webserver started at http://127.30.8.62:33977/ using document root <none> and password file <none>
I20260811 15:12:38.687047 30713 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:38.687129 30713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:38.687335 30713 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:38.689577 30713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "ff12bc92fe12424e9881a41899d69970"
format_stamp: "Formatted at 2026-08-11 15:12:38 on dist-test-slave-1vmg"
server_key: "94342e17a35370ae7d57e4edcd47ded5"
server_key_iv: "75277468952aa82412b0fcabf3d456b1"
server_key_version: "encryptionkey@0"
I20260811 15:12:38.690143 30713 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "ff12bc92fe12424e9881a41899d69970"
format_stamp: "Formatted at 2026-08-11 15:12:38 on dist-test-slave-1vmg"
server_key: "94342e17a35370ae7d57e4edcd47ded5"
server_key_iv: "75277468952aa82412b0fcabf3d456b1"
server_key_version: "encryptionkey@0"
I20260811 15:12:38.694164 30713 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260811 15:12:38.696833 30739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:38.698163 30713 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.002s
I20260811 15:12:38.698292 30713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "ff12bc92fe12424e9881a41899d69970"
format_stamp: "Formatted at 2026-08-11 15:12:38 on dist-test-slave-1vmg"
server_key: "94342e17a35370ae7d57e4edcd47ded5"
server_key_iv: "75277468952aa82412b0fcabf3d456b1"
server_key_version: "encryptionkey@0"
I20260811 15:12:38.698413 30713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:38.707932 30713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:38.708626 30713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:38.708817 30713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:38.716024 30713 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:37125
I20260811 15:12:38.716050 30829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:37125 every 8 connection(s)
I20260811 15:12:38.717022 30713 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:12:38.720396 30830 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:38.724280 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 30713
I20260811 15:12:38.724400 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:12:38.724648 30752 external_mini_cluster.cc:1428] Setting key be1e043d89795a84577dcec7e76df4ff
I20260811 15:12:38.726785 30830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970: Bootstrap starting.
I20260811 15:12:38.729346 30830 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:38.730257 30830 log.cc:824] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:38.732412 30830 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970: No bootstrap required, opened a new log
I20260811 15:12:38.734426 30837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:38.726185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50776 (local address 127.30.8.62:37125)
0811 15:12:38.726833 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:38.726850 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:38.726891 (+ 41us) server_negotiation.cc:407] Connection header received
0811 15:12:38.727605 (+ 714us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:38.727623 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:38.728034 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:38.728319 (+ 285us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:38.728750 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.729775 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.731001 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.731492 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.731663 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:38.732622 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:38.732628 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:38.732643 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:38.733055 (+ 412us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:38.733119 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:38.733141 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:38.733243 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:38.733406 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 15:12:38.733686 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":119,"threads_started":1}
I20260811 15:12:38.736444 30830 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff12bc92fe12424e9881a41899d69970" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 37125 } }
I20260811 15:12:38.736693 30830 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:38.736757 30830 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff12bc92fe12424e9881a41899d69970, State: Initialized, Role: FOLLOWER
I20260811 15:12:38.737192 30830 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff12bc92fe12424e9881a41899d69970" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 37125 } }
I20260811 15:12:38.737315 30830 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:12:38.737361 30830 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:12:38.737432 30830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:38.738402 30830 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff12bc92fe12424e9881a41899d69970" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 37125 } }
I20260811 15:12:38.738775 30830 leader_election.cc:302] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff12bc92fe12424e9881a41899d69970; no voters:
I20260811 15:12:38.739136 30830 leader_election.cc:288] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:12:38.739250 30840 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:38.739467 30840 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970 [term 1 LEADER]: Becoming Leader. State: Replica: ff12bc92fe12424e9881a41899d69970, State: Running, Role: LEADER
I20260811 15:12:38.739785 30840 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff12bc92fe12424e9881a41899d69970" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 37125 } }
I20260811 15:12:38.740378 30830 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:12:38.741482 30842 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff12bc92fe12424e9881a41899d69970. Latest consensus state: current_term: 1 leader_uuid: "ff12bc92fe12424e9881a41899d69970" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff12bc92fe12424e9881a41899d69970" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 37125 } } }
I20260811 15:12:38.741489 30841 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff12bc92fe12424e9881a41899d69970" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff12bc92fe12424e9881a41899d69970" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 37125 } } }
I20260811 15:12:38.741622 30842 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970 [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:38.741634 30841 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970 [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:38.742133 30852 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:12:38.745908 30852 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:12:38.757167 30852 catalog_manager.cc:1347] Generated new cluster ID: 40cbafa150d0455b95a86aed0ac24e6f
I20260811 15:12:38.757263 30852 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:12:38.783114 30852 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:12:38.784132 30852 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:12:38.791203 30852 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970: Generated new TSK 0
I20260811 15:12:38.791970 30852 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:12:38.799806 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.1:0
--local_ip_for_outbound_sockets=127.30.8.1
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:37125
--builtin_ntp_servers=127.30.8.20:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 15:12:38.919461 30865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:38.919780 30865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:38.919821 30865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:38.923947 30865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:38.924024 30865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:38.924098 30865 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.1
I20260811 15:12:38.929024 30865 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.1:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.8.1
--webserver_port=0
--tserver_master_addrs=127.30.8.62:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.1
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:38.930240 30865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:38.931727 30865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:38.941658 30874 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:38.941602 30879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:38.941753 30865 server_base.cc:1034] running on GCE node
W20260811 15:12:38.941602 30881 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:38.942289 30865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:38.943004 30865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:38.944180 30865 hybrid_clock.cc:648] HybridClock initialized: now 1786461158944160 us; error 38 us; skew 500 ppm
I20260811 15:12:38.946110 30865 webserver.cc:460] Webserver started at http://127.30.8.1:44055/ using document root <none> and password file <none>
I20260811 15:12:38.946722 30865 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:38.946794 30865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:38.947057 30865 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:38.949330 30865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-0/data/instance:
uuid: "947e45db464441eabafacea25268ffb6"
format_stamp: "Formatted at 2026-08-11 15:12:38 on dist-test-slave-1vmg"
server_key: "f757d65ed009743c899d17fd83ef95db"
server_key_iv: "3bbc8ba22f64855be0b8a97e06e32d4a"
server_key_version: "encryptionkey@0"
I20260811 15:12:38.949908 30865 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance:
uuid: "947e45db464441eabafacea25268ffb6"
format_stamp: "Formatted at 2026-08-11 15:12:38 on dist-test-slave-1vmg"
server_key: "f757d65ed009743c899d17fd83ef95db"
server_key_iv: "3bbc8ba22f64855be0b8a97e06e32d4a"
server_key_version: "encryptionkey@0"
I20260811 15:12:38.953730 30865 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 15:12:38.956240 30891 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:38.957415 30865 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 15:12:38.957544 30865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-0/wal
uuid: "947e45db464441eabafacea25268ffb6"
format_stamp: "Formatted at 2026-08-11 15:12:38 on dist-test-slave-1vmg"
server_key: "f757d65ed009743c899d17fd83ef95db"
server_key_iv: "3bbc8ba22f64855be0b8a97e06e32d4a"
server_key_version: "encryptionkey@0"
I20260811 15:12:38.957648 30865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:38.979413 30865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:38.980203 30865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:38.980393 30865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:38.981033 30865 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:38.982061 30865 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:12:38.982124 30865 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:38.982183 30865 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:12:38.982208 30865 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:38.991576 30865 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.1:44307
I20260811 15:12:38.991608 31065 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.1:44307 every 8 connection(s)
I20260811 15:12:38.992542 30865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-0/data/info.pb
I20260811 15:12:38.998334 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 30865
I20260811 15:12:38.998493 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-0/wal/instance
I20260811 15:12:38.998781 30752 external_mini_cluster.cc:1428] Setting key dd7dfc74fa235e16a3b73dd7a9c5bff1
I20260811 15:12:39.001034 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.2:0
--local_ip_for_outbound_sockets=127.30.8.2
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:37125
--builtin_ntp_servers=127.30.8.20:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 15:12:39.002396 30837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:38.994363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.1:58851 (local address 127.30.8.62:37125)
0811 15:12:38.994545 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:38.994549 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:38.995647 (+ 1098us) server_negotiation.cc:407] Connection header received
0811 15:12:38.997059 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:38.997062 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:38.997111 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:38.997185 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:38.998146 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.998752 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.000279 (+ 1527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.000577 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.000701 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:39.001903 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:39.001908 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:39.001911 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:39.001927 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:39.001954 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:39.001965 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:39.002052 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:39.002170 (+ 118us) server_negotiation.cc:299] Negotiation successful
0811 15:12:39.002232 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 15:12:39.003355 31070 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:38.994776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:37125 (local address 127.30.8.1:58851)
0811 15:12:38.995495 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:38.995539 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:38.996849 (+ 1310us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:38.997347 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:38.997354 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:38.997442 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:38.997972 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:38.998002 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:38.998895 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:38.998899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:39.000079 (+ 1180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:39.000094 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.000740 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.000743 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:39.000898 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:39.001613 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:12:39.001615 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:39.001661 (+ 46us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:12:39.001735 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:39.002048 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:39.002053 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:39.002057 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 15:12:39.002150 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 15:12:39.002444 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":146,"threads_started":1}
I20260811 15:12:39.004627 31067 heartbeater.cc:347] Connected to a master server at 127.30.8.62:37125
I20260811 15:12:39.004907 31067 heartbeater.cc:464] Registering TS with master...
I20260811 15:12:39.005580 31067 heartbeater.cc:511] Master 127.30.8.62:37125 requested a full tablet report, sending...
I20260811 15:12:39.008424 30776 ts_manager.cc:194] Registered new tserver with Master: 947e45db464441eabafacea25268ffb6 (127.30.8.1:44307)
I20260811 15:12:39.010051 30776 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.8.1:58851
W20260811 15:12:39.121459 31072 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:39.121799 31072 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:39.121841 31072 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:39.126142 31072 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:39.126251 31072 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:39.126315 31072 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.2
I20260811 15:12:39.131458 31072 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.2:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.8.2
--webserver_port=0
--tserver_master_addrs=127.30.8.62:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31072
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.2
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:39.132740 31072 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:39.134234 31072 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:39.146831 31082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:39.146799 31085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:39.146806 31078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:39.147220 31072 server_base.cc:1034] running on GCE node
I20260811 15:12:39.147707 31072 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:39.148492 31072 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:39.149674 31072 hybrid_clock.cc:648] HybridClock initialized: now 1786461159149655 us; error 45 us; skew 500 ppm
I20260811 15:12:39.151765 31072 webserver.cc:460] Webserver started at http://127.30.8.2:42599/ using document root <none> and password file <none>
I20260811 15:12:39.152384 31072 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:39.152449 31072 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:39.152661 31072 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:39.154814 31072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-1/data/instance:
uuid: "9011a93a917641bf973b6a49a28ba88f"
format_stamp: "Formatted at 2026-08-11 15:12:39 on dist-test-slave-1vmg"
server_key: "65297d28b1cc2dc6f2567587bdb10bf9"
server_key_iv: "8f85193d1159923492002a105efe6da0"
server_key_version: "encryptionkey@0"
I20260811 15:12:39.155417 31072 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance:
uuid: "9011a93a917641bf973b6a49a28ba88f"
format_stamp: "Formatted at 2026-08-11 15:12:39 on dist-test-slave-1vmg"
server_key: "65297d28b1cc2dc6f2567587bdb10bf9"
server_key_iv: "8f85193d1159923492002a105efe6da0"
server_key_version: "encryptionkey@0"
I20260811 15:12:39.159224 31072 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:12:39.161809 31092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:39.163168 31072 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:12:39.163285 31072 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-1/wal
uuid: "9011a93a917641bf973b6a49a28ba88f"
format_stamp: "Formatted at 2026-08-11 15:12:39 on dist-test-slave-1vmg"
server_key: "65297d28b1cc2dc6f2567587bdb10bf9"
server_key_iv: "8f85193d1159923492002a105efe6da0"
server_key_version: "encryptionkey@0"
I20260811 15:12:39.163396 31072 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:39.173566 31072 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:39.174248 31072 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:39.174419 31072 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:39.175057 31072 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:39.176555 31072 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:12:39.176647 31072 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:39.176723 31072 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:12:39.176759 31072 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:39.188252 31072 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.2:40357
I20260811 15:12:39.188304 31283 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.2:40357 every 8 connection(s)
I20260811 15:12:39.189680 31072 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-1/data/info.pb
I20260811 15:12:39.200029 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 31072
I20260811 15:12:39.200202 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-1/wal/instance
I20260811 15:12:39.200248 30837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:39.192058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.2:39211 (local address 127.30.8.62:37125)
0811 15:12:39.192212 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:39.192217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:39.193449 (+ 1232us) server_negotiation.cc:407] Connection header received
0811 15:12:39.195061 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:39.195065 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:39.195146 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:39.195265 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:39.196369 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.196935 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.198176 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.198400 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.198479 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:39.199719 (+ 1240us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:39.199721 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:39.199724 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:39.199750 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:39.199766 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:39.199773 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:39.199827 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:39.200025 (+ 198us) server_negotiation.cc:299] Negotiation successful
0811 15:12:39.200086 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 15:12:39.200568 30752 external_mini_cluster.cc:1428] Setting key 4f0357029be607ecd87c5fad979b21d3
I20260811 15:12:39.201349 31287 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:39.192488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:37125 (local address 127.30.8.2:39211)
0811 15:12:39.193277 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:39.193324 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:39.194696 (+ 1372us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:39.195454 (+ 758us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:39.195462 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:39.195556 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:39.196125 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:39.196140 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.197073 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.197076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:39.198022 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:39.198034 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.198555 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.198559 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:39.198721 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:39.199447 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:12:39.199449 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:39.199496 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:12:39.199573 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:39.199848 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:39.199857 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:39.199865 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 15:12:39.200034 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 15:12:39.200362 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":166,"threads_started":1}
I20260811 15:12:39.202739 31284 heartbeater.cc:347] Connected to a master server at 127.30.8.62:37125
I20260811 15:12:39.203025 31284 heartbeater.cc:464] Registering TS with master...
I20260811 15:12:39.203065 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.8.3:0
--local_ip_for_outbound_sockets=127.30.8.3
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:37125
--builtin_ntp_servers=127.30.8.20:34895
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 15:12:39.203682 31284 heartbeater.cc:511] Master 127.30.8.62:37125 requested a full tablet report, sending...
I20260811 15:12:39.204871 30776 ts_manager.cc:194] Registered new tserver with Master: 9011a93a917641bf973b6a49a28ba88f (127.30.8.2:40357)
I20260811 15:12:39.205539 30776 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.8.2:39211
W20260811 15:12:39.325506 31288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:39.325852 31288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:39.325917 31288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:39.331900 31288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:39.331979 31288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:39.332070 31288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.8.3
I20260811 15:12:39.337235 31288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:34895
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.8.3:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.8.3
--webserver_port=0
--tserver_master_addrs=127.30.8.62:37125
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.8.3
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:39.338461 31288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:39.340018 31288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:39.350533 31299 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:39.350559 31296 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:39.350559 31297 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:39.350761 31288 server_base.cc:1034] running on GCE node
I20260811 15:12:39.351347 31288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:39.352089 31288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:39.353348 31288 hybrid_clock.cc:648] HybridClock initialized: now 1786461159353301 us; error 34 us; skew 500 ppm
I20260811 15:12:39.356604 31288 webserver.cc:460] Webserver started at http://127.30.8.3:44791/ using document root <none> and password file <none>
I20260811 15:12:39.357517 31288 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:39.357627 31288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:39.357869 31288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:39.361042 31288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-2/data/instance:
uuid: "976bf4d604974ee794999902cb9282a4"
format_stamp: "Formatted at 2026-08-11 15:12:39 on dist-test-slave-1vmg"
server_key: "d1c55ecb4f721cecb648cd299b6aa3a6"
server_key_iv: "c09b09f603a82e089d7961cc878d2c2b"
server_key_version: "encryptionkey@0"
I20260811 15:12:39.362552 31288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance:
uuid: "976bf4d604974ee794999902cb9282a4"
format_stamp: "Formatted at 2026-08-11 15:12:39 on dist-test-slave-1vmg"
server_key: "d1c55ecb4f721cecb648cd299b6aa3a6"
server_key_iv: "c09b09f603a82e089d7961cc878d2c2b"
server_key_version: "encryptionkey@0"
I20260811 15:12:39.367514 31288 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 15:12:39.370586 31314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:39.372742 31288 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 15:12:39.372948 31288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-2/wal
uuid: "976bf4d604974ee794999902cb9282a4"
format_stamp: "Formatted at 2026-08-11 15:12:39 on dist-test-slave-1vmg"
server_key: "d1c55ecb4f721cecb648cd299b6aa3a6"
server_key_iv: "c09b09f603a82e089d7961cc878d2c2b"
server_key_version: "encryptionkey@0"
I20260811 15:12:39.373131 31288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:39.393090 31288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:39.393898 31288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:39.394088 31288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:39.394749 31288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 15:12:39.395857 31288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 15:12:39.395915 31288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:39.395972 31288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 15:12:39.395998 31288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:39.406213 31288 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.3:34139
I20260811 15:12:39.406265 31503 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.3:34139 every 8 connection(s)
I20260811 15:12:39.407353 31288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-2/data/info.pb
I20260811 15:12:39.413003 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 31288
I20260811 15:12:39.413120 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-2/wal/instance
I20260811 15:12:39.413375 30752 external_mini_cluster.cc:1428] Setting key fbef74e1655836c69c62e703b140898c
I20260811 15:12:39.417598 30837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:39.409394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:60473 (local address 127.30.8.62:37125)
0811 15:12:39.409550 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:39.409555 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:39.410711 (+ 1156us) server_negotiation.cc:407] Connection header received
0811 15:12:39.412282 (+ 1571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:39.412285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:39.412348 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:39.412430 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:39.413559 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.414231 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.415629 (+ 1398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.415910 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.416017 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:39.417155 (+ 1138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:39.417158 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:39.417160 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:39.417174 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:39.417197 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:39.417206 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:39.417273 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:39.417400 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 15:12:39.417462 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 15:12:39.418562 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:39.409813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:37125 (local address 127.30.8.3:60473)
0811 15:12:39.410544 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:39.410591 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:39.412023 (+ 1432us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:39.412629 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:39.412636 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:39.412736 (+ 100us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:12:39.413341 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:39.413359 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.414377 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.414381 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:39.415492 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:39.415504 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.416070 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.416073 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:39.416247 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:39.416872 (+ 625us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 15:12:39.416873 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:12:39.416920 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 15:12:39.416999 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:12:39.417277 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:12:39.417283 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:12:39.417287 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 15:12:39.417388 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 15:12:39.417654 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":120,"threads_started":1}
I20260811 15:12:39.419996 31504 heartbeater.cc:347] Connected to a master server at 127.30.8.62:37125
I20260811 15:12:39.420256 31504 heartbeater.cc:464] Registering TS with master...
I20260811 15:12:39.420838 31504 heartbeater.cc:511] Master 127.30.8.62:37125 requested a full tablet report, sending...
I20260811 15:12:39.421950 30776 ts_manager.cc:194] Registered new tserver with Master: 976bf4d604974ee794999902cb9282a4 (127.30.8.3:34139)
I20260811 15:12:39.422547 30776 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.8.3:60473
I20260811 15:12:39.427541 30752 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 15:12:39.433069 30837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:39.429582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50788 (local address 127.30.8.62:37125)
0811 15:12:39.429700 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:39.429705 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:39.429863 (+ 158us) server_negotiation.cc:407] Connection header received
0811 15:12:39.429977 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:39.429979 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:39.430021 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:39.430087 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:39.430308 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.430823 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.431688 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.431900 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.431979 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:39.432724 (+ 745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:39.432728 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:39.432730 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:39.432744 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:39.432763 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:39.432771 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:39.432827 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:39.432885 (+ 58us) server_negotiation.cc:299] Negotiation successful
0811 15:12:39.432944 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 15:12:39.436563 30776 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50788:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 15:12:39.439802 30776 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 15:12:39.455076 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:39.449025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53714 (local address 127.30.8.2:40357)
0811 15:12:39.449486 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:39.449500 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:39.449908 (+ 408us) server_negotiation.cc:407] Connection header received
0811 15:12:39.450168 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:39.450175 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:39.450401 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:39.450975 (+ 574us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:39.451118 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.452275 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.453719 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.454296 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.454407 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:39.454661 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:39.454754 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 15:12:39.454872 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":357,"thread_start_us":131,"threads_started":1}
I20260811 15:12:39.455076 31522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:39.448912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:40357 (local address 127.0.0.1:53714)
0811 15:12:39.449817 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:39.449830 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:39.449968 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:39.450605 (+ 637us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:39.450612 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:39.450646 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:39.450801 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:39.450806 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.452411 (+ 1605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.452415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:39.453572 (+ 1157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:39.453585 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.454428 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.454432 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:39.454543 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:39.454582 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 15:12:39.454708 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 15:12:39.454857 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":41,"spinlock_wait_cycles":4608,"thread_start_us":100,"threads_started":1}
I20260811 15:12:39.455435 31525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:39.449672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.3:34139 (local address 127.0.0.1:51518)
0811 15:12:39.450551 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:39.450564 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:39.450653 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:39.451245 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:39.451249 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:39.451269 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:39.451351 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:39.451357 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.452680 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.452682 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:39.453587 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:39.453598 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.454976 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.454981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:39.455073 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:39.455100 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 15:12:39.455183 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 15:12:39.455264 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":808,"mutex_wait_us":23,"spinlock_wait_cycles":6016,"thread_start_us":108,"threads_started":1}
I20260811 15:12:39.455952 31528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:39.450189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51518 (local address 127.30.8.3:34139)
0811 15:12:39.450630 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:39.450641 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:39.450661 (+ 20us) server_negotiation.cc:407] Connection header received
0811 15:12:39.450775 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:39.450781 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:39.450977 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:39.451111 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:39.451482 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.452559 (+ 1077us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.453731 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.454562 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.454682 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:39.454881 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:39.455648 (+ 767us) server_negotiation.cc:299] Negotiation successful
0811 15:12:39.455771 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":154,"threads_started":1}
I20260811 15:12:39.456840 31524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:39.449250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:44307 (local address 127.0.0.1:53278)
0811 15:12:39.449751 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:39.449808 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:39.449944 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:39.450968 (+ 1024us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:39.450971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:39.450993 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:39.451103 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:39.451110 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.452350 (+ 1240us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.452353 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:39.454703 (+ 2350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:39.454715 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.456379 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.456384 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:39.456493 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:39.456521 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 15:12:39.456607 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 15:12:39.456692 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":106,"threads_started":1}
I20260811 15:12:39.457460 31526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:39.450149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53278 (local address 127.30.8.1:44307)
0811 15:12:39.450488 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:39.450500 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:39.450523 (+ 23us) server_negotiation.cc:407] Connection header received
0811 15:12:39.450600 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:39.450604 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:39.450795 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:39.450992 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:39.451199 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.452196 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.454867 (+ 2671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.455934 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.456096 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:39.456305 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:39.457177 (+ 872us) server_negotiation.cc:299] Negotiation successful
0811 15:12:39.457293 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":110,"threads_started":1}
I20260811 15:12:39.458796 31399 tablet_service.cc:1467] Processing CreateTablet for tablet 0bb1cdc26c4841c4b88e684bca23cf54 (DEFAULT_TABLE table=test-table [id=63ea2f8e2bf24337ada21ec3a08e09f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:12:39.458806 31184 tablet_service.cc:1467] Processing CreateTablet for tablet 0bb1cdc26c4841c4b88e684bca23cf54 (DEFAULT_TABLE table=test-table [id=63ea2f8e2bf24337ada21ec3a08e09f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:12:39.459937 31399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bb1cdc26c4841c4b88e684bca23cf54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:39.459956 31184 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bb1cdc26c4841c4b88e684bca23cf54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:39.460124 30967 tablet_service.cc:1467] Processing CreateTablet for tablet 0bb1cdc26c4841c4b88e684bca23cf54 (DEFAULT_TABLE table=test-table [id=63ea2f8e2bf24337ada21ec3a08e09f3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 15:12:39.461241 30967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bb1cdc26c4841c4b88e684bca23cf54. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:39.467327 31531 tablet_bootstrap.cc:492] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4: Bootstrap starting.
I20260811 15:12:39.469144 31532 tablet_bootstrap.cc:492] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f: Bootstrap starting.
I20260811 15:12:39.469481 31533 tablet_bootstrap.cc:492] T 0bb1cdc26c4841c4b88e684bca23cf54 P 947e45db464441eabafacea25268ffb6: Bootstrap starting.
I20260811 15:12:39.470582 31531 tablet_bootstrap.cc:654] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:39.471323 31532 tablet_bootstrap.cc:654] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:39.471638 31531 log.cc:824] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:39.471939 31533 tablet_bootstrap.cc:654] T 0bb1cdc26c4841c4b88e684bca23cf54 P 947e45db464441eabafacea25268ffb6: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:39.472151 31532 log.cc:824] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:39.472930 31533 log.cc:824] T 0bb1cdc26c4841c4b88e684bca23cf54 P 947e45db464441eabafacea25268ffb6: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:39.474516 31532 tablet_bootstrap.cc:492] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f: No bootstrap required, opened a new log
I20260811 15:12:39.474735 31532 ts_tablet_manager.cc:1397] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 15:12:39.475018 31531 tablet_bootstrap.cc:492] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4: No bootstrap required, opened a new log
I20260811 15:12:39.475116 31533 tablet_bootstrap.cc:492] T 0bb1cdc26c4841c4b88e684bca23cf54 P 947e45db464441eabafacea25268ffb6: No bootstrap required, opened a new log
I20260811 15:12:39.475267 31533 ts_tablet_manager.cc:1397] T 0bb1cdc26c4841c4b88e684bca23cf54 P 947e45db464441eabafacea25268ffb6: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 15:12:39.475267 31531 ts_tablet_manager.cc:1397] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.002s
I20260811 15:12:39.478528 31532 raft_consensus.cc:348] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9011a93a917641bf973b6a49a28ba88f" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40357 } } peers { permanent_uuid: "976bf4d604974ee794999902cb9282a4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 34139 } } peers { permanent_uuid: "947e45db464441eabafacea25268ffb6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44307 } }
I20260811 15:12:39.478801 31532 raft_consensus.cc:374] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:39.478861 31532 raft_consensus.cc:729] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9011a93a917641bf973b6a49a28ba88f, State: Initialized, Role: FOLLOWER
I20260811 15:12:39.479400 31532 consensus_queue.cc:260] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9011a93a917641bf973b6a49a28ba88f" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40357 } } peers { permanent_uuid: "976bf4d604974ee794999902cb9282a4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 34139 } } peers { permanent_uuid: "947e45db464441eabafacea25268ffb6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44307 } }
I20260811 15:12:39.480217 31531 raft_consensus.cc:348] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9011a93a917641bf973b6a49a28ba88f" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40357 } } peers { permanent_uuid: "976bf4d604974ee794999902cb9282a4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 34139 } } peers { permanent_uuid: "947e45db464441eabafacea25268ffb6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44307 } }
I20260811 15:12:39.480217 31533 raft_consensus.cc:348] T 0bb1cdc26c4841c4b88e684bca23cf54 P 947e45db464441eabafacea25268ffb6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9011a93a917641bf973b6a49a28ba88f" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40357 } } peers { permanent_uuid: "976bf4d604974ee794999902cb9282a4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 34139 } } peers { permanent_uuid: "947e45db464441eabafacea25268ffb6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44307 } }
I20260811 15:12:39.480556 31531 raft_consensus.cc:374] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:39.480556 31533 raft_consensus.cc:374] T 0bb1cdc26c4841c4b88e684bca23cf54 P 947e45db464441eabafacea25268ffb6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:39.480610 31533 raft_consensus.cc:729] T 0bb1cdc26c4841c4b88e684bca23cf54 P 947e45db464441eabafacea25268ffb6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 947e45db464441eabafacea25268ffb6, State: Initialized, Role: FOLLOWER
I20260811 15:12:39.480621 31531 raft_consensus.cc:729] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 976bf4d604974ee794999902cb9282a4, State: Initialized, Role: FOLLOWER
I20260811 15:12:39.480665 31532 ts_tablet_manager.cc:1428] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f: Time spent starting tablet: real 0.006s user 0.002s sys 0.004s
I20260811 15:12:39.481092 31284 heartbeater.cc:503] Master 127.30.8.62:37125 was elected leader, sending a full tablet report...
I20260811 15:12:39.481169 31533 consensus_queue.cc:260] T 0bb1cdc26c4841c4b88e684bca23cf54 P 947e45db464441eabafacea25268ffb6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9011a93a917641bf973b6a49a28ba88f" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40357 } } peers { permanent_uuid: "976bf4d604974ee794999902cb9282a4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 34139 } } peers { permanent_uuid: "947e45db464441eabafacea25268ffb6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44307 } }
I20260811 15:12:39.481169 31531 consensus_queue.cc:260] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9011a93a917641bf973b6a49a28ba88f" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40357 } } peers { permanent_uuid: "976bf4d604974ee794999902cb9282a4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 34139 } } peers { permanent_uuid: "947e45db464441eabafacea25268ffb6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44307 } }
I20260811 15:12:39.482297 31067 heartbeater.cc:503] Master 127.30.8.62:37125 was elected leader, sending a full tablet report...
I20260811 15:12:39.482378 31531 ts_tablet_manager.cc:1428] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260811 15:12:39.482405 31504 heartbeater.cc:503] Master 127.30.8.62:37125 was elected leader, sending a full tablet report...
I20260811 15:12:39.482677 31533 ts_tablet_manager.cc:1428] T 0bb1cdc26c4841c4b88e684bca23cf54 P 947e45db464441eabafacea25268ffb6: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 15:12:39.484512 31543 raft_consensus.cc:482] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 15:12:39.484699 31543 raft_consensus.cc:504] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9011a93a917641bf973b6a49a28ba88f" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40357 } } peers { permanent_uuid: "976bf4d604974ee794999902cb9282a4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 34139 } } peers { permanent_uuid: "947e45db464441eabafacea25268ffb6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44307 } }
I20260811 15:12:39.487203 31543 leader_election.cc:288] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9011a93a917641bf973b6a49a28ba88f (127.30.8.2:40357), 947e45db464441eabafacea25268ffb6 (127.30.8.1:44307)
I20260811 15:12:39.490818 31510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:39.486343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.2:40357 (local address 127.30.8.3:35989)
0811 15:12:39.486576 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:39.486591 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:39.486681 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:39.487152 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:39.487156 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:39.487176 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:39.487319 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:39.487327 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.488311 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.488314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:39.489525 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:39.489537 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.490341 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.490346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:39.490491 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:39.490525 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 15:12:39.490618 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 15:12:39.490704 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146}
I20260811 15:12:39.491025 31523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:39.486604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:35989 (local address 127.30.8.2:40357)
0811 15:12:39.486767 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:39.486772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:39.486787 (+ 15us) server_negotiation.cc:407] Connection header received
0811 15:12:39.486846 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:39.486849 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:39.486901 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:39.487005 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:39.487475 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.488174 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.489659 (+ 1485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.490205 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.490305 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:39.490381 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:39.490688 (+ 307us) server_negotiation.cc:299] Negotiation successful
0811 15:12:39.490846 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 15:12:39.491665 31214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bb1cdc26c4841c4b88e684bca23cf54" candidate_uuid: "976bf4d604974ee794999902cb9282a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9011a93a917641bf973b6a49a28ba88f" is_pre_election: true
I20260811 15:12:39.492057 31214 raft_consensus.cc:2454] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 976bf4d604974ee794999902cb9282a4 in term 0.
I20260811 15:12:39.492606 31322 leader_election.cc:302] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9011a93a917641bf973b6a49a28ba88f, 976bf4d604974ee794999902cb9282a4; no voters:
I20260811 15:12:39.492959 31543 raft_consensus.cc:2781] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 15:12:39.493046 31543 raft_consensus.cc:482] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 15:12:39.493080 31543 raft_consensus.cc:3037] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [term 0 FOLLOWER]: Advancing to term 1
W20260811 15:12:39.493681 31068 tablet.cc:2369] T 0bb1cdc26c4841c4b88e684bca23cf54 P 947e45db464441eabafacea25268ffb6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:12:39.494035 31526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:39.486730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.8.3:58945 (local address 127.30.8.1:44307)
0811 15:12:39.487601 (+ 871us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:39.487606 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:39.487622 (+ 16us) server_negotiation.cc:407] Connection header received
0811 15:12:39.490054 (+ 2432us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:39.490059 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:39.490119 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:39.490208 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 15:12:39.490642 (+ 434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.491413 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.492869 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.493400 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.493513 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:39.493581 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:39.493840 (+ 259us) server_negotiation.cc:299] Negotiation successful
0811 15:12:39.493924 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":778}
I20260811 15:12:39.494052 31544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:39.486354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:44307 (local address 127.30.8.3:58945)
0811 15:12:39.487531 (+ 1177us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:39.487551 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 15:12:39.489894 (+ 2343us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:12:39.490236 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:12:39.490239 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:12:39.490257 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 15:12:39.490438 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:39.490445 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.491548 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.491552 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:39.492682 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:12:39.492693 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.493563 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.493567 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:12:39.493670 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:39.493702 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 15:12:39.493792 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 15:12:39.493871 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1071,"mutex_wait_us":464,"thread_start_us":193,"threads_started":1}
I20260811 15:12:39.494513 31543 raft_consensus.cc:504] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9011a93a917641bf973b6a49a28ba88f" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40357 } } peers { permanent_uuid: "976bf4d604974ee794999902cb9282a4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 34139 } } peers { permanent_uuid: "947e45db464441eabafacea25268ffb6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44307 } }
I20260811 15:12:39.494835 31001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bb1cdc26c4841c4b88e684bca23cf54" candidate_uuid: "976bf4d604974ee794999902cb9282a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947e45db464441eabafacea25268ffb6" is_pre_election: true
I20260811 15:12:39.494949 31543 leader_election.cc:288] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [CANDIDATE]: Term 1 election: Requested vote from peers 9011a93a917641bf973b6a49a28ba88f (127.30.8.2:40357), 947e45db464441eabafacea25268ffb6 (127.30.8.1:44307)
I20260811 15:12:39.495184 31001 raft_consensus.cc:2454] T 0bb1cdc26c4841c4b88e684bca23cf54 P 947e45db464441eabafacea25268ffb6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 976bf4d604974ee794999902cb9282a4 in term 0.
I20260811 15:12:39.495342 31000 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bb1cdc26c4841c4b88e684bca23cf54" candidate_uuid: "976bf4d604974ee794999902cb9282a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "947e45db464441eabafacea25268ffb6"
I20260811 15:12:39.495460 31000 raft_consensus.cc:3037] T 0bb1cdc26c4841c4b88e684bca23cf54 P 947e45db464441eabafacea25268ffb6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:39.495673 31214 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bb1cdc26c4841c4b88e684bca23cf54" candidate_uuid: "976bf4d604974ee794999902cb9282a4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9011a93a917641bf973b6a49a28ba88f"
I20260811 15:12:39.495817 31214 raft_consensus.cc:3037] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:39.496825 31000 raft_consensus.cc:2454] T 0bb1cdc26c4841c4b88e684bca23cf54 P 947e45db464441eabafacea25268ffb6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 976bf4d604974ee794999902cb9282a4 in term 1.
I20260811 15:12:39.497078 31214 raft_consensus.cc:2454] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 976bf4d604974ee794999902cb9282a4 in term 1.
I20260811 15:12:39.497335 31318 leader_election.cc:302] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 947e45db464441eabafacea25268ffb6, 976bf4d604974ee794999902cb9282a4; no voters:
I20260811 15:12:39.497521 31543 raft_consensus.cc:2781] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:39.497764 31543 raft_consensus.cc:686] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [term 1 LEADER]: Becoming Leader. State: Replica: 976bf4d604974ee794999902cb9282a4, State: Running, Role: LEADER
I20260811 15:12:39.498088 31543 consensus_queue.cc:237] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9011a93a917641bf973b6a49a28ba88f" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40357 } } peers { permanent_uuid: "976bf4d604974ee794999902cb9282a4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 34139 } } peers { permanent_uuid: "947e45db464441eabafacea25268ffb6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44307 } }
I20260811 15:12:39.501475 30776 catalog_manager.cc:5495] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 976bf4d604974ee794999902cb9282a4 (127.30.8.3). New cstate: current_term: 1 leader_uuid: "976bf4d604974ee794999902cb9282a4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9011a93a917641bf973b6a49a28ba88f" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "976bf4d604974ee794999902cb9282a4" member_type: VOTER last_known_addr { host: "127.30.8.3" port: 34139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "947e45db464441eabafacea25268ffb6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44307 } health_report { overall_health: UNKNOWN } } }
I20260811 15:12:39.521354 31528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:39.518058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51526 (local address 127.30.8.3:34139)
0811 15:12:39.518180 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:39.518184 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:39.518197 (+ 13us) server_negotiation.cc:407] Connection header received
0811 15:12:39.518289 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:39.518291 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:39.518349 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:39.518452 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:12:39.518638 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.519183 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.520034 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:39.520259 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:39.520343 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:39.520574 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:12:39.521062 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:12:39.521154 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:39.521180 (+ 26us) server_negotiation.cc:299] Negotiation successful
0811 15:12:39.521234 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 15:12:39.527429 31214 raft_consensus.cc:1261] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f [term 1 FOLLOWER]: Refusing update from remote peer 976bf4d604974ee794999902cb9282a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:12:39.527434 31000 raft_consensus.cc:1261] T 0bb1cdc26c4841c4b88e684bca23cf54 P 947e45db464441eabafacea25268ffb6 [term 1 FOLLOWER]: Refusing update from remote peer 976bf4d604974ee794999902cb9282a4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:12:39.528131 31543 consensus_queue.cc:1035] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "947e45db464441eabafacea25268ffb6" member_type: VOTER last_known_addr { host: "127.30.8.1" port: 44307 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:12:39.528322 31545 consensus_queue.cc:1035] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9011a93a917641bf973b6a49a28ba88f" member_type: VOTER last_known_addr { host: "127.30.8.2" port: 40357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:12:39.536320 31556 mvcc.cc:205] Tried to move back new op lower bound from 7317344909416972288 to 7317344909306908672. Current Snapshot: MvccSnapshot[applied={T|T < 7317344909416972288}]
I20260811 15:12:39.537170 31553 mvcc.cc:205] Tried to move back new op lower bound from 7317344909416972288 to 7317344909306908672. Current Snapshot: MvccSnapshot[applied={T|T < 7317344909416972288}]
I20260811 15:12:39.545320 30772 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:12:39.545492 30772 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50788:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 15:12:39.548091 30772 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ff12bc92fe12424e9881a41899d69970: Sending DeleteTablet for 3 replicas of tablet 0bb1cdc26c4841c4b88e684bca23cf54
I20260811 15:12:39.549230 30967 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bb1cdc26c4841c4b88e684bca23cf54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:12:39 UTC) from {username='slave'} at 127.0.0.1:53278
I20260811 15:12:39.549233 31184 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bb1cdc26c4841c4b88e684bca23cf54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:12:39 UTC) from {username='slave'} at 127.0.0.1:53714
I20260811 15:12:39.549790 31399 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bb1cdc26c4841c4b88e684bca23cf54 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 15:12:39 UTC) from {username='slave'} at 127.0.0.1:51518
I20260811 15:12:39.549829 31574 tablet_replica.cc:330] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f: stopping tablet replica
I20260811 15:12:39.550199 31575 tablet_replica.cc:330] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4: stopping tablet replica
I20260811 15:12:39.550191 31574 raft_consensus.cc:2229] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 15:12:39.550338 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 30865
I20260811 15:12:39.550555 31574 raft_consensus.cc:2258] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:12:39.555662 31574 ts_tablet_manager.cc:1905] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:12:39.558079 31574 ts_tablet_manager.cc:1918] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:12:39.558179 31574 log.cc:1196] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-1/wal/wals/0bb1cdc26c4841c4b88e684bca23cf54
I20260811 15:12:39.558439 31574 ts_tablet_manager.cc:1939] T 0bb1cdc26c4841c4b88e684bca23cf54 P 9011a93a917641bf973b6a49a28ba88f: Deleting consensus metadata
I20260811 15:12:39.559787 30745 catalog_manager.cc:4841] TS 9011a93a917641bf973b6a49a28ba88f (127.30.8.2:40357): tablet 0bb1cdc26c4841c4b88e684bca23cf54 (table test-table [id=63ea2f8e2bf24337ada21ec3a08e09f3]) successfully deleted
I20260811 15:12:39.560942 31575 raft_consensus.cc:2229] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [term 1 LEADER]: Raft consensus shutting down.
I20260811 15:12:39.561688 31575 raft_consensus.cc:2258] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 15:12:39.564000 31575 ts_tablet_manager.cc:1905] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 15:12:39.567590 31575 ts_tablet_manager.cc:1918] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 15:12:39.567718 31575 log.cc:1196] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4: Deleting WAL directory at /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786460893564915-30752-0/minicluster-data/ts-2/wal/wals/0bb1cdc26c4841c4b88e684bca23cf54
I20260811 15:12:39.568043 31575 ts_tablet_manager.cc:1939] T 0bb1cdc26c4841c4b88e684bca23cf54 P 976bf4d604974ee794999902cb9282a4: Deleting consensus metadata
I20260811 15:12:39.570091 30745 catalog_manager.cc:4841] TS 976bf4d604974ee794999902cb9282a4 (127.30.8.3:34139): tablet 0bb1cdc26c4841c4b88e684bca23cf54 (table test-table [id=63ea2f8e2bf24337ada21ec3a08e09f3]) successfully deleted
W20260811 15:12:39.570112 30741 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.8.1:44307 (error 108)
I20260811 15:12:39.570483 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 31072
I20260811 15:12:39.575304 31524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:39.574811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.1:44307 (local address 127.0.0.1:53284)
0811 15:12:39.575117 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 15:12:39.575199 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.1:44307: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218}
W20260811 15:12:39.575522 30741 catalog_manager.cc:4584] TS 947e45db464441eabafacea25268ffb6 (127.30.8.1:44307): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0bb1cdc26c4841c4b88e684bca23cf54: Network error: Client connection negotiation failed: client connection to 127.30.8.1:44307: connect: Connection refused (error 111)
I20260811 15:12:39.587960 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 31288
I20260811 15:12:39.608328 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 30713
2026-08-11T15:12:39Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1110 ms)
[----------] 6 tests from EncryptionPolicyTest (11350 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:12:39 dist-test-slave-1vmg krb5kdc[31582](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:12:39 dist-test-slave-1vmg krb5kdc[31582](info): set up 2 sockets
Aug 11 15:12:39 dist-test-slave-1vmg krb5kdc[31582](info): commencing operation
krb5kdc: starting...
W20260811 15:12:40.818655 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.182s user 0.009s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:12:40 dist-test-slave-1vmg krb5kdc[31582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:12:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:12:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 15:12:40.958117 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36289
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36289
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:41.099534 31601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:41.099954 31601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:41.100028 31601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:41.104450 31601 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:12:41.104553 31601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:41.104583 31601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:41.104605 31601 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:12:41.104626 31601 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:12:41.110872 31601 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36289
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:41.112493 31601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:41.113873 31601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:41.124691 31617 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:41.124691 31610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:41.124696 31612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:41.125094 31601 server_base.cc:1034] running on GCE node
I20260811 15:12:41.125806 31601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:41.127238 31601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:41.128401 31601 hybrid_clock.cc:648] HybridClock initialized: now 1786461161128383 us; error 32 us; skew 500 ppm
Aug 11 15:12:41 dist-test-slave-1vmg krb5kdc[31582](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461161, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:41.131407 31601 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 15:12:41.132507 31601 webserver.cc:460] Webserver started at http://127.0.0.1:33835/ using document root <none> and password file <none>
I20260811 15:12:41.133147 31601 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:41.133229 31601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:41.133479 31601 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:41.135876 31601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "b014442521474913944079a29b39bee7"
format_stamp: "Formatted at 2026-08-11 15:12:41 on dist-test-slave-1vmg"
server_key: "a74621ae61a10833265743ffd0eea355"
server_key_iv: "c36128998f1f3c29f969d7bdcd897e73"
server_key_version: "encryptionkey@0"
I20260811 15:12:41.136492 31601 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "b014442521474913944079a29b39bee7"
format_stamp: "Formatted at 2026-08-11 15:12:41 on dist-test-slave-1vmg"
server_key: "a74621ae61a10833265743ffd0eea355"
server_key_iv: "c36128998f1f3c29f969d7bdcd897e73"
server_key_version: "encryptionkey@0"
I20260811 15:12:41.140739 31601 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:12:41.143427 31632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:41.144788 31601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:12:41.144922 31601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "b014442521474913944079a29b39bee7"
format_stamp: "Formatted at 2026-08-11 15:12:41 on dist-test-slave-1vmg"
server_key: "a74621ae61a10833265743ffd0eea355"
server_key_iv: "c36128998f1f3c29f969d7bdcd897e73"
server_key_version: "encryptionkey@0"
I20260811 15:12:41.145037 31601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:41.169492 31601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:41.172775 31601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:41.172991 31601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:41.181413 31601 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36289
I20260811 15:12:41.181430 31724 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36289 every 8 connection(s)
I20260811 15:12:41.182530 31601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:12:41.185813 31728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:41.188261 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 31601
I20260811 15:12:41.188390 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:12:41.188675 30752 external_mini_cluster.cc:1428] Setting key 8d6c0b844b8b22190c7d69d5fac4897f
I20260811 15:12:41.192487 31728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7: Bootstrap starting.
I20260811 15:12:41.194887 31728 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:41.195756 31728 log.cc:824] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7: Log is configured to *not* fsync() on all Append() calls
Aug 11 15:12:41 dist-test-slave-1vmg krb5kdc[31582](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461160, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 15:12:41.198014 31728 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7: No bootstrap required, opened a new log
I20260811 15:12:41.201900 31728 raft_consensus.cc:348] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b014442521474913944079a29b39bee7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36289 } }
I20260811 15:12:41.202171 31728 raft_consensus.cc:374] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:41.202239 31728 raft_consensus.cc:729] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b014442521474913944079a29b39bee7, State: Initialized, Role: FOLLOWER
I20260811 15:12:41.202701 31728 consensus_queue.cc:260] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b014442521474913944079a29b39bee7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36289 } }
I20260811 15:12:41.202857 31728 raft_consensus.cc:388] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:12:41.202946 31728 raft_consensus.cc:482] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:12:41.203059 31728 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:41.203382 31736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:41.190196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46731 (local address 127.0.0.1:36289)
0811 15:12:41.190695 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:41.190711 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:41.190750 (+ 39us) server_negotiation.cc:407] Connection header received
0811 15:12:41.191489 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:41.191509 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:41.191878 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:41.192225 (+ 347us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:41.193083 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:41.194099 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:41.194970 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:41.195487 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:41.195655 (+ 168us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:41.198372 (+ 2717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:41.198378 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:41.198394 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:41.198423 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:41.200124 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:41.200640 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:41.200645 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:41.200650 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:41.200784 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:41.201139 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:41.201142 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:41.201143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:41.201578 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:41.201793 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:41.202138 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 15:12:41.202402 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":156,"threads_started":1}
I20260811 15:12:41.204301 31728 raft_consensus.cc:504] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b014442521474913944079a29b39bee7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36289 } }
I20260811 15:12:41.204701 31728 leader_election.cc:302] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b014442521474913944079a29b39bee7; no voters:
I20260811 15:12:41.205053 31728 leader_election.cc:288] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:12:41.205159 31738 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:41.205334 31738 raft_consensus.cc:686] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7 [term 1 LEADER]: Becoming Leader. State: Replica: b014442521474913944079a29b39bee7, State: Running, Role: LEADER
I20260811 15:12:41.206001 31738 consensus_queue.cc:237] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b014442521474913944079a29b39bee7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36289 } }
I20260811 15:12:41.206492 31728 sys_catalog.cc:564] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:12:41.206651 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:12:41.207942 31739 sys_catalog.cc:455] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b014442521474913944079a29b39bee7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b014442521474913944079a29b39bee7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36289 } } }
I20260811 15:12:41.208117 31739 sys_catalog.cc:458] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7 [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:41.207937 31740 sys_catalog.cc:455] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b014442521474913944079a29b39bee7. Latest consensus state: current_term: 1 leader_uuid: "b014442521474913944079a29b39bee7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b014442521474913944079a29b39bee7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36289 } } }
I20260811 15:12:41.208341 31740 sys_catalog.cc:458] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7 [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:41.208683 31768 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:12:41.213099 31768 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:12:41.217324 31736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:41.209362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53197 (local address 127.0.0.1:36289)
0811 15:12:41.209475 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:41.209480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:41.209710 (+ 230us) server_negotiation.cc:407] Connection header received
0811 15:12:41.209828 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:41.209830 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:41.209881 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:41.209978 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:41.210786 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:41.211360 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:41.212162 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:41.212385 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:41.212474 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:41.213902 (+ 1428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:41.213905 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:41.213907 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:41.213932 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:41.215309 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:41.215903 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:41.215907 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:41.215909 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:41.216001 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:41.216353 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:41.216356 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:41.216357 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:41.216583 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:41.216732 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:41.217056 (+ 324us) server_negotiation.cc:299] Negotiation successful
0811 15:12:41.217173 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260811 15:12:41.219693 31768 catalog_manager.cc:1347] Generated new cluster ID: 6f7aff86385841be9f1f0219a016cd90
I20260811 15:12:41.219837 31768 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:12:41.231791 31768 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:12:41.233180 31768 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:12:41.238513 31768 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b014442521474913944079a29b39bee7: Generated new TSK 0
I20260811 15:12:41.239456 31768 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:12:41.246125 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 31601
W20260811 15:12:41.259720 31579 connection.cc:535] client connection to 127.0.0.1:36289 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36289): Connection reset by peer (error 104)
2026-08-11T15:12:41Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1637 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:12:41 dist-test-slave-1vmg krb5kdc[31782](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:12:41 dist-test-slave-1vmg krb5kdc[31782](info): set up 2 sockets
Aug 11 15:12:41 dist-test-slave-1vmg krb5kdc[31782](info): commencing operation
krb5kdc: starting...
W20260811 15:12:42.444727 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.170s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:12:42 dist-test-slave-1vmg krb5kdc[31782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461162, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:12:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:12:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 15:12:42.596495 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34231
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34231
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:42.732404 31808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:42.732734 31808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:42.732795 31808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:42.737035 31808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:12:42.737128 31808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:42.737156 31808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:42.737178 31808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:12:42.737207 31808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:12:42.742988 31808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34231
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34231
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:42.744326 31808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:42.745541 31808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:42.754797 31815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:42.754798 31816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:42.754798 31819 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:42.755247 31808 server_base.cc:1034] running on GCE node
I20260811 15:12:42.755877 31808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:42.757427 31808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:42.758685 31808 hybrid_clock.cc:648] HybridClock initialized: now 1786461162758645 us; error 63 us; skew 500 ppm
Aug 11 15:12:42 dist-test-slave-1vmg krb5kdc[31782](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461162, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:42.762099 31808 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 15:12:42.763592 31808 webserver.cc:460] Webserver started at http://127.0.0.1:38281/ using document root <none> and password file <none>
I20260811 15:12:42.764369 31808 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:42.764449 31808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:42.764772 31808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:42.767643 31808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "7a16ad1cdb2d4a09a8bb92070ff5fcf4"
format_stamp: "Formatted at 2026-08-11 15:12:42 on dist-test-slave-1vmg"
server_key: "87731a82a6d469b0233c0ee4bc1b2ec9"
server_key_iv: "c7d06a9c68c4d8cbaeb567e5abaea4de"
server_key_version: "encryptionkey@0"
I20260811 15:12:42.768366 31808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "7a16ad1cdb2d4a09a8bb92070ff5fcf4"
format_stamp: "Formatted at 2026-08-11 15:12:42 on dist-test-slave-1vmg"
server_key: "87731a82a6d469b0233c0ee4bc1b2ec9"
server_key_iv: "c7d06a9c68c4d8cbaeb567e5abaea4de"
server_key_version: "encryptionkey@0"
I20260811 15:12:42.773459 31808 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 15:12:42.777061 31827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:42.778820 31808 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260811 15:12:42.779068 31808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "7a16ad1cdb2d4a09a8bb92070ff5fcf4"
format_stamp: "Formatted at 2026-08-11 15:12:42 on dist-test-slave-1vmg"
server_key: "87731a82a6d469b0233c0ee4bc1b2ec9"
server_key_iv: "c7d06a9c68c4d8cbaeb567e5abaea4de"
server_key_version: "encryptionkey@0"
I20260811 15:12:42.779215 31808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:42.789954 31808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:42.793709 31808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:42.793946 31808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:42.802875 31808 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34231
I20260811 15:12:42.802946 31911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34231 every 8 connection(s)
I20260811 15:12:42.804169 31808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:12:42.806145 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 31808
I20260811 15:12:42.806322 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:12:42.806669 30752 external_mini_cluster.cc:1428] Setting key ad5930a88cfe439a091624ce963104e3
I20260811 15:12:42.807936 31915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:42.815629 31915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4: Bootstrap starting.
I20260811 15:12:42.817539 31919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:42.808661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57915 (local address 127.0.0.1:34231)
0811 15:12:42.809206 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:42.809229 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:42.809277 (+ 48us) server_negotiation.cc:407] Connection header received
0811 15:12:42.810088 (+ 811us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:42.810111 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:42.810549 (+ 438us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:42.810954 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:42.811705 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:42.812815 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:42.814042 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:42.814562 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:42.814687 (+ 125us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:42.815678 (+ 991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:42.815683 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:42.815699 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:42.816167 (+ 468us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:42.816233 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:42.816260 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:42.816368 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:42.816560 (+ 192us) server_negotiation.cc:299] Negotiation successful
0811 15:12:42.816710 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":172,"threads_started":1}
I20260811 15:12:42.818302 31915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:42.819168 31915 log.cc:824] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:42.820871 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:12:42.821507 31915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4: No bootstrap required, opened a new log
I20260811 15:12:42.827345 31915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a16ad1cdb2d4a09a8bb92070ff5fcf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34231 } }
I20260811 15:12:42.827709 31915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:42.827782 31915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7a16ad1cdb2d4a09a8bb92070ff5fcf4, State: Initialized, Role: FOLLOWER
I20260811 15:12:42.828161 31919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:42.823370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40649 (local address 127.0.0.1:34231)
0811 15:12:42.823476 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:42.823480 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:42.823733 (+ 253us) server_negotiation.cc:407] Connection header received
0811 15:12:42.823911 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:42.823913 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:42.823955 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:42.824029 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:42.824427 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:42.824927 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:42.826355 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:42.826595 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:42.826682 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:42.827596 (+ 914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:42.827599 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:42.827601 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:42.827614 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:42.827641 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:42.827652 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:42.827730 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:42.827961 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 15:12:42.828001 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260811 15:12:42.828416 31915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a16ad1cdb2d4a09a8bb92070ff5fcf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34231 } }
I20260811 15:12:42.828583 31915 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:12:42.828655 31915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:12:42.828759 31915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:42.830073 31915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a16ad1cdb2d4a09a8bb92070ff5fcf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34231 } }
I20260811 15:12:42.830489 31915 leader_election.cc:302] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7a16ad1cdb2d4a09a8bb92070ff5fcf4; no voters:
I20260811 15:12:42.830926 31915 leader_election.cc:288] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:12:42.831794 31932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:42.832020 31932 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4 [term 1 LEADER]: Becoming Leader. State: Replica: 7a16ad1cdb2d4a09a8bb92070ff5fcf4, State: Running, Role: LEADER
I20260811 15:12:42.832352 31932 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a16ad1cdb2d4a09a8bb92070ff5fcf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34231 } }
I20260811 15:12:42.832502 31915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:12:42.834134 31929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7a16ad1cdb2d4a09a8bb92070ff5fcf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a16ad1cdb2d4a09a8bb92070ff5fcf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34231 } } }
I20260811 15:12:42.834266 31929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4 [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:42.834654 31933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7a16ad1cdb2d4a09a8bb92070ff5fcf4. Latest consensus state: current_term: 1 leader_uuid: "7a16ad1cdb2d4a09a8bb92070ff5fcf4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7a16ad1cdb2d4a09a8bb92070ff5fcf4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34231 } } }
I20260811 15:12:42.834753 31933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4 [sys.catalog]: This master's current role is: LEADER
W20260811 15:12:42.837522 31961 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:12:42.837603 31961 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:12:42.837679 31958 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:12:42.838376 31958 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:12:42.844357 31958 catalog_manager.cc:1347] Generated new cluster ID: f5ebafeaea934cdbaa57d1ba20d911f3
I20260811 15:12:42.844448 31958 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:12:42.855468 31958 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:12:42.856487 31958 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:12:42.863348 31958 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7a16ad1cdb2d4a09a8bb92070ff5fcf4: Generated new TSK 0
I20260811 15:12:42.864135 31958 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:12:42.880316 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 31808
W20260811 15:12:42.891824 31780 connection.cc:535] client connection to 127.0.0.1:34231 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34231): Connection reset by peer (error 104)
2026-08-11T15:12:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1632 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:12:42 dist-test-slave-1vmg krb5kdc[31973](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:12:42 dist-test-slave-1vmg krb5kdc[31973](info): set up 2 sockets
Aug 11 15:12:42 dist-test-slave-1vmg krb5kdc[31973](info): commencing operation
krb5kdc: starting...
W20260811 15:12:44.070530 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.164s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:12:44 dist-test-slave-1vmg krb5kdc[31973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461164, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:12:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:12:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 15:12:44.215456 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38555
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38555
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:44.341275 31998 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:44.341612 31998 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:44.341677 31998 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:44.345893 31998 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:12:44.345983 31998 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:44.346010 31998 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:44.346031 31998 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:12:44.346052 31998 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:12:44.351758 31998 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38555
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38555
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31998
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:44.353015 31998 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:44.354095 31998 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:44.362883 32006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:44.362890 32007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:44.362887 32010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:44.363179 31998 server_base.cc:1034] running on GCE node
I20260811 15:12:44.363867 31998 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:44.365070 31998 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:44.366232 31998 hybrid_clock.cc:648] HybridClock initialized: now 1786461164366210 us; error 39 us; skew 500 ppm
Aug 11 15:12:44 dist-test-slave-1vmg krb5kdc[31973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461164, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:44.373641 31998 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 15:12:44.374810 31998 webserver.cc:460] Webserver started at http://127.0.0.1:36677/ using document root <none> and password file <none>
I20260811 15:12:44.375494 31998 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:44.375572 31998 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:44.375821 31998 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:44.378065 31998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "76c3074edf814ab19646d5d309deb60e"
format_stamp: "Formatted at 2026-08-11 15:12:44 on dist-test-slave-1vmg"
server_key: "ca23f4fa14247819caaa4e0e96394f40"
server_key_iv: "d517bdd0cac43285122ba00014509fa8"
server_key_version: "encryptionkey@0"
I20260811 15:12:44.378643 31998 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "76c3074edf814ab19646d5d309deb60e"
format_stamp: "Formatted at 2026-08-11 15:12:44 on dist-test-slave-1vmg"
server_key: "ca23f4fa14247819caaa4e0e96394f40"
server_key_iv: "d517bdd0cac43285122ba00014509fa8"
server_key_version: "encryptionkey@0"
I20260811 15:12:44.382853 31998 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 15:12:44.386005 32023 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:44.387539 31998 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 15:12:44.387710 31998 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "76c3074edf814ab19646d5d309deb60e"
format_stamp: "Formatted at 2026-08-11 15:12:44 on dist-test-slave-1vmg"
server_key: "ca23f4fa14247819caaa4e0e96394f40"
server_key_iv: "d517bdd0cac43285122ba00014509fa8"
server_key_version: "encryptionkey@0"
I20260811 15:12:44.387835 31998 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:44.392961 31998 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:44.397403 31998 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:44.397662 31998 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:44.407213 31998 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38555
I20260811 15:12:44.407253 32109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38555 every 8 connection(s)
I20260811 15:12:44.408488 31998 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:12:44.411887 32110 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:44.415057 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 31998
I20260811 15:12:44.415232 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:12:44.415514 30752 external_mini_cluster.cc:1428] Setting key e009ded03e0e5233e0806424bc13656a
I20260811 15:12:44.418265 32110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e: Bootstrap starting.
I20260811 15:12:44.420848 32113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:44.416916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58913 (local address 127.0.0.1:38555)
0811 15:12:44.417379 (+ 463us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:44.417396 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:44.417435 (+ 39us) server_negotiation.cc:407] Connection header received
0811 15:12:44.418088 (+ 653us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:44.418105 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:44.418432 (+ 327us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:44.418741 (+ 309us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:44.419621 (+ 880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:44.419627 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:44.419638 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:44.419649 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:44.419706 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:44.419739 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:44.419849 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:44.419942 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 15:12:44.420094 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":146,"threads_started":1}
I20260811 15:12:44.420934 32110 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:44.421851 32110 log.cc:824] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:44.423919 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:12:44.424095 32110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e: No bootstrap required, opened a new log
I20260811 15:12:44.427654 32113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:44.426001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49469 (local address 127.0.0.1:38555)
0811 15:12:44.426125 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:44.426129 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:44.426259 (+ 130us) server_negotiation.cc:407] Connection header received
0811 15:12:44.426386 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:44.426389 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:44.426426 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:44.426491 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:44.427283 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:44.427287 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:44.427290 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:44.427302 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:44.427333 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:44.427346 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:44.427435 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:44.427483 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 15:12:44.427520 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 15:12:44.428239 32110 raft_consensus.cc:348] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c3074edf814ab19646d5d309deb60e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38555 } }
I20260811 15:12:44.428483 32110 raft_consensus.cc:374] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:44.428540 32110 raft_consensus.cc:729] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 76c3074edf814ab19646d5d309deb60e, State: Initialized, Role: FOLLOWER
I20260811 15:12:44.428978 32110 consensus_queue.cc:260] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c3074edf814ab19646d5d309deb60e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38555 } }
I20260811 15:12:44.429136 32110 raft_consensus.cc:388] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:12:44.429201 32110 raft_consensus.cc:482] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:12:44.429304 32110 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:44.430269 32110 raft_consensus.cc:504] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c3074edf814ab19646d5d309deb60e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38555 } }
I20260811 15:12:44.430619 32110 leader_election.cc:302] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 76c3074edf814ab19646d5d309deb60e; no voters:
I20260811 15:12:44.430945 32110 leader_election.cc:288] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:12:44.431095 32127 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:44.431308 32127 raft_consensus.cc:686] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e [term 1 LEADER]: Becoming Leader. State: Replica: 76c3074edf814ab19646d5d309deb60e, State: Running, Role: LEADER
I20260811 15:12:44.431605 32127 consensus_queue.cc:237] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c3074edf814ab19646d5d309deb60e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38555 } }
I20260811 15:12:44.432255 32110 sys_catalog.cc:564] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:12:44.433264 32130 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "76c3074edf814ab19646d5d309deb60e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c3074edf814ab19646d5d309deb60e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38555 } } }
I20260811 15:12:44.433400 32130 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:44.433697 32132 sys_catalog.cc:455] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 76c3074edf814ab19646d5d309deb60e. Latest consensus state: current_term: 1 leader_uuid: "76c3074edf814ab19646d5d309deb60e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "76c3074edf814ab19646d5d309deb60e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38555 } } }
I20260811 15:12:44.433794 32132 sys_catalog.cc:458] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e [sys.catalog]: This master's current role is: LEADER
W20260811 15:12:44.437355 32153 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:12:44.437491 32153 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:12:44.437628 32144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:12:44.438565 32144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:12:44.445024 32144 catalog_manager.cc:1347] Generated new cluster ID: 27f3ff1941024ef0bc9fe67b2da86a96
I20260811 15:12:44.445116 32144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:12:44.472426 32144 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:12:44.472797 32144 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:12:44.484396 32144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 76c3074edf814ab19646d5d309deb60e: Generated new TSK 0
I20260811 15:12:44.485275 32144 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:12:44.491000 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 31998
2026-08-11T15:12:44Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1616 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:12:44 dist-test-slave-1vmg krb5kdc[32167](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:12:44 dist-test-slave-1vmg krb5kdc[32167](info): set up 2 sockets
Aug 11 15:12:44 dist-test-slave-1vmg krb5kdc[32167](info): commencing operation
krb5kdc: starting...
W20260811 15:12:45.699071 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.176s user 0.006s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:12:45 dist-test-slave-1vmg krb5kdc[32167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:12:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:12:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 15:12:45.846705 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41109
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42995
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41109
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:45.966281 32194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:45.966601 32194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:45.966646 32194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:45.970887 32194 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:12:45.971026 32194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:45.971048 32194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:45.971079 32194 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:12:45.971099 32194 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:12:45.976909 32194 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42995
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41109
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41109
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:45.978168 32194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:45.979277 32194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:45.988214 32205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:45.988214 32200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:45.988493 32201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:45.988516 32194 server_base.cc:1034] running on GCE node
I20260811 15:12:45.989355 32194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:45.990633 32194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:45.991827 32194 hybrid_clock.cc:648] HybridClock initialized: now 1786461165991793 us; error 64 us; skew 500 ppm
Aug 11 15:12:45 dist-test-slave-1vmg krb5kdc[32167](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461165, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:45.995535 32194 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 15:12:45.996937 32194 webserver.cc:460] Webserver started at http://127.0.0.1:37533/ using document root <none> and password file <none>
I20260811 15:12:45.997720 32194 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:45.997831 32194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:45.998065 32194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:46.000608 32194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "8376c6abf5d84bf58f64d3c19f0c7355"
format_stamp: "Formatted at 2026-08-11 15:12:45 on dist-test-slave-1vmg"
server_key: "ecd9520d145e1a9c4402aff1ac95de22"
server_key_iv: "4993fb56779550a00d2a5dfc920baf10"
server_key_version: "encryptionkey@0"
I20260811 15:12:46.001219 32194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "8376c6abf5d84bf58f64d3c19f0c7355"
format_stamp: "Formatted at 2026-08-11 15:12:45 on dist-test-slave-1vmg"
server_key: "ecd9520d145e1a9c4402aff1ac95de22"
server_key_iv: "4993fb56779550a00d2a5dfc920baf10"
server_key_version: "encryptionkey@0"
I20260811 15:12:46.005396 32194 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:12:46.008126 32217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:46.009370 32194 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:12:46.009518 32194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "8376c6abf5d84bf58f64d3c19f0c7355"
format_stamp: "Formatted at 2026-08-11 15:12:45 on dist-test-slave-1vmg"
server_key: "ecd9520d145e1a9c4402aff1ac95de22"
server_key_iv: "4993fb56779550a00d2a5dfc920baf10"
server_key_version: "encryptionkey@0"
I20260811 15:12:46.009631 32194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:46.023823 32194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:46.027022 32194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:46.027230 32194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:46.034663 32194 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41109
I20260811 15:12:46.034687 32312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41109 every 8 connection(s)
I20260811 15:12:46.035821 32194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:12:46.038844 32313 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:46.044855 32313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355: Bootstrap starting.
I20260811 15:12:46.045632 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 32194
I20260811 15:12:46.045768 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:12:46.046027 30752 external_mini_cluster.cc:1428] Setting key c6f378273e7430b66e2885db86bff408
I20260811 15:12:46.047838 32313 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:46.048797 32313 log.cc:824] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:46.051182 32313 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355: No bootstrap required, opened a new log
Aug 11 15:12:46 dist-test-slave-1vmg krb5kdc[32167](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461165, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 15:12:46.055593 32313 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8376c6abf5d84bf58f64d3c19f0c7355" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } }
I20260811 15:12:46.055922 32313 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:46.055996 32313 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8376c6abf5d84bf58f64d3c19f0c7355, State: Initialized, Role: FOLLOWER
I20260811 15:12:46.056505 32313 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8376c6abf5d84bf58f64d3c19f0c7355" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } }
I20260811 15:12:46.056694 32313 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:12:46.056772 32313 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:12:46.056885 32313 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:46.058159 32313 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8376c6abf5d84bf58f64d3c19f0c7355" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } }
I20260811 15:12:46.058573 32313 leader_election.cc:302] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8376c6abf5d84bf58f64d3c19f0c7355; no voters:
I20260811 15:12:46.058979 32313 leader_election.cc:288] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:12:46.059101 32320 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:46.059337 32320 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355 [term 1 LEADER]: Becoming Leader. State: Replica: 8376c6abf5d84bf58f64d3c19f0c7355, State: Running, Role: LEADER
I20260811 15:12:46.059773 32320 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8376c6abf5d84bf58f64d3c19f0c7355" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } }
I20260811 15:12:46.060500 32313 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:12:46.061735 32325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8376c6abf5d84bf58f64d3c19f0c7355" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8376c6abf5d84bf58f64d3c19f0c7355" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } } }
I20260811 15:12:46.061717 32323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8376c6abf5d84bf58f64d3c19f0c7355. Latest consensus state: current_term: 1 leader_uuid: "8376c6abf5d84bf58f64d3c19f0c7355" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8376c6abf5d84bf58f64d3c19f0c7355" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41109 } } }
I20260811 15:12:46.061864 32323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355 [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:46.061864 32325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355 [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:46.062002 32318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:46.047356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54709 (local address 127.0.0.1:41109)
0811 15:12:46.047827 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:46.047847 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:46.047893 (+ 46us) server_negotiation.cc:407] Connection header received
0811 15:12:46.048809 (+ 916us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:46.048836 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:46.049305 (+ 469us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:46.049610 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:46.050336 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:46.051790 (+ 1454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:46.052701 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:46.053430 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:46.053606 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:46.056458 (+ 2852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:46.056465 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:46.056484 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:46.056522 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:46.058504 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:46.059011 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:46.059018 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:46.059025 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:46.059185 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:46.059505 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:46.059508 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:46.059510 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:46.060068 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:46.060284 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:46.060621 (+ 337us) server_negotiation.cc:299] Negotiation successful
0811 15:12:46.060953 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":130,"threads_started":1}
I20260811 15:12:46.065053 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260811 15:12:46.065713 32346 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:12:46.065784 32346 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:12:46.066519 32333 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:12:46.067523 32333 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:12:46.074003 32333 catalog_manager.cc:1347] Generated new cluster ID: 0c52326be8424042a66c64479d379331
I20260811 15:12:46.074129 32333 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:12:46.075932 32318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:46.067614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60255 (local address 127.0.0.1:41109)
0811 15:12:46.067725 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:46.067729 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:46.067754 (+ 25us) server_negotiation.cc:407] Connection header received
0811 15:12:46.067790 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:46.067793 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:46.067833 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:46.067894 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:46.068561 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:46.069078 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:46.070215 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:46.070570 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:46.070681 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:46.072263 (+ 1582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:46.072267 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:46.072270 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:46.072297 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:46.073956 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:46.074551 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:46.074555 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:46.074557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:46.074653 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:46.075007 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:46.075011 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:46.075013 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:46.075258 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:46.075388 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:46.075632 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 15:12:46.075765 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 15:12:46.087278 32333 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:12:46.088393 32333 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:12:46.096539 32333 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8376c6abf5d84bf58f64d3c19f0c7355: Generated new TSK 0
I20260811 15:12:46.097402 32333 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:12:46.113108 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 32194
2026-08-11T15:12:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1621 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:12:46 dist-test-slave-1vmg krb5kdc[32363](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:12:46 dist-test-slave-1vmg krb5kdc[32363](info): set up 2 sockets
Aug 11 15:12:46 dist-test-slave-1vmg krb5kdc[32363](info): commencing operation
krb5kdc: starting...
W20260811 15:12:47.331264 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.187s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:12:47 dist-test-slave-1vmg krb5kdc[32363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:12:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:12:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 15:12:47.482246 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35349
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35349
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:47.615080 32378 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:47.615403 32378 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:47.615447 32378 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:47.619611 32378 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:12:47.619688 32378 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:47.619709 32378 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:47.619724 32378 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:12:47.619760 32378 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:12:47.625437 32378 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35349
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32378
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:47.626685 32378 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:47.627801 32378 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 15:12:47.637244 32378 server_base.cc:1034] running on GCE node
W20260811 15:12:47.637180 32392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:47.637153 32394 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:47.637173 32389 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:47.638489 32378 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:47.640208 32378 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:47.641431 32378 hybrid_clock.cc:648] HybridClock initialized: now 1786461167641387 us; error 65 us; skew 500 ppm
Aug 11 15:12:47 dist-test-slave-1vmg krb5kdc[32363](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461167, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:47.644860 32378 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 15:12:47.645938 32378 webserver.cc:460] Webserver started at http://127.0.0.1:41119/ using document root <none> and password file <none>
I20260811 15:12:47.646531 32378 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:47.646595 32378 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:47.646791 32378 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:47.649026 32378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "74e47a7cd30e4223bf969bc63e383381"
format_stamp: "Formatted at 2026-08-11 15:12:47 on dist-test-slave-1vmg"
server_key: "4b31fea2bebabb98ae318da92879e5df"
server_key_iv: "7c1a42aa9468b7256a3d0307773ec726"
server_key_version: "encryptionkey@0"
I20260811 15:12:47.649593 32378 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "74e47a7cd30e4223bf969bc63e383381"
format_stamp: "Formatted at 2026-08-11 15:12:47 on dist-test-slave-1vmg"
server_key: "4b31fea2bebabb98ae318da92879e5df"
server_key_iv: "7c1a42aa9468b7256a3d0307773ec726"
server_key_version: "encryptionkey@0"
I20260811 15:12:47.653703 32378 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 15:12:47.656406 32404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:47.658424 32378 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 15:12:47.658658 32378 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "74e47a7cd30e4223bf969bc63e383381"
format_stamp: "Formatted at 2026-08-11 15:12:47 on dist-test-slave-1vmg"
server_key: "4b31fea2bebabb98ae318da92879e5df"
server_key_iv: "7c1a42aa9468b7256a3d0307773ec726"
server_key_version: "encryptionkey@0"
I20260811 15:12:47.658831 32378 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:47.688102 32378 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:47.692093 32378 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:47.692310 32378 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:47.699615 32378 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35349
I20260811 15:12:47.699648 32496 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35349 every 8 connection(s)
I20260811 15:12:47.700711 32378 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:12:47.701306 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 32378
I20260811 15:12:47.701532 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:12:47.701889 30752 external_mini_cluster.cc:1428] Setting key 611bd488949091b2841ba7830253cff5
I20260811 15:12:47.704212 32497 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:47.711169 32497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381: Bootstrap starting.
I20260811 15:12:47.712662 32501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:47.703742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50767 (local address 127.0.0.1:35349)
0811 15:12:47.704322 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:47.704349 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:47.704414 (+ 65us) server_negotiation.cc:407] Connection header received
0811 15:12:47.705281 (+ 867us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:47.705305 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:47.705779 (+ 474us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:47.706159 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:47.706588 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:47.707785 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:47.709149 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:47.709700 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:47.709845 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:47.710756 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:47.710762 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:47.710778 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:47.711268 (+ 490us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:47.711341 (+ 73us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:47.711378 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:47.711464 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:47.711618 (+ 154us) server_negotiation.cc:299] Negotiation successful
0811 15:12:47.711862 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":172,"threads_started":1}
I20260811 15:12:47.713857 32497 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:47.714619 32497 log.cc:824] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:47.716179 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:12:47.716756 32497 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381: No bootstrap required, opened a new log
I20260811 15:12:47.721561 32497 raft_consensus.cc:348] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e47a7cd30e4223bf969bc63e383381" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35349 } }
I20260811 15:12:47.722035 32497 raft_consensus.cc:374] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:47.722096 32497 raft_consensus.cc:729] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 74e47a7cd30e4223bf969bc63e383381, State: Initialized, Role: FOLLOWER
I20260811 15:12:47.722783 32497 consensus_queue.cc:260] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e47a7cd30e4223bf969bc63e383381" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35349 } }
I20260811 15:12:47.723004 32497 raft_consensus.cc:388] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:12:47.723093 32497 raft_consensus.cc:482] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:12:47.723227 32497 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:47.723807 32501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:47.719097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52325 (local address 127.0.0.1:35349)
0811 15:12:47.719284 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:47.719288 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:47.719557 (+ 269us) server_negotiation.cc:407] Connection header received
0811 15:12:47.719742 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:47.719745 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:47.719796 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:47.719864 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:47.720229 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:47.720782 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:47.721955 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:47.722252 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:47.722338 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:47.723372 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:47.723376 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:47.723378 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:47.723392 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:47.723420 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:47.723431 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:47.723497 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:47.723614 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 15:12:47.723673 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 15:12:47.724745 32497 raft_consensus.cc:504] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e47a7cd30e4223bf969bc63e383381" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35349 } }
I20260811 15:12:47.725204 32497 leader_election.cc:302] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 74e47a7cd30e4223bf969bc63e383381; no voters:
I20260811 15:12:47.725713 32497 leader_election.cc:288] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:12:47.725865 32511 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:47.726150 32511 raft_consensus.cc:686] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381 [term 1 LEADER]: Becoming Leader. State: Replica: 74e47a7cd30e4223bf969bc63e383381, State: Running, Role: LEADER
I20260811 15:12:47.726576 32511 consensus_queue.cc:237] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e47a7cd30e4223bf969bc63e383381" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35349 } }
I20260811 15:12:47.727210 32497 sys_catalog.cc:564] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:12:47.728402 32513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 74e47a7cd30e4223bf969bc63e383381. Latest consensus state: current_term: 1 leader_uuid: "74e47a7cd30e4223bf969bc63e383381" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e47a7cd30e4223bf969bc63e383381" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35349 } } }
I20260811 15:12:47.728565 32513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381 [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:47.728974 32521 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:12:47.728945 32512 sys_catalog.cc:455] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "74e47a7cd30e4223bf969bc63e383381" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "74e47a7cd30e4223bf969bc63e383381" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35349 } } }
I20260811 15:12:47.729190 32512 sys_catalog.cc:458] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381 [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:47.732101 32521 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:12:47.738118 32521 catalog_manager.cc:1347] Generated new cluster ID: db3583f49e404622afa8955d50c3867b
I20260811 15:12:47.738210 32521 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:12:47.769126 32521 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:12:47.770682 32521 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:12:47.782584 32521 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 74e47a7cd30e4223bf969bc63e383381: Generated new TSK 0
I20260811 15:12:47.783449 32521 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:12:47.795621 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 32378
2026-08-11T15:12:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1680 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:12:47 dist-test-slave-1vmg krb5kdc[32542](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:12:47 dist-test-slave-1vmg krb5kdc[32542](info): set up 2 sockets
Aug 11 15:12:47 dist-test-slave-1vmg krb5kdc[32542](info): commencing operation
krb5kdc: starting...
W20260811 15:12:48.991937 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.167s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:12:49 dist-test-slave-1vmg krb5kdc[32542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:12:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:12:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 15:12:49.134724 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44611
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44611
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:49.266669 32563 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:49.267041 32563 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:49.267107 32563 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:49.271323 32563 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:12:49.271409 32563 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:49.271428 32563 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:49.271459 32563 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:12:49.271487 32563 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:12:49.277403 32563 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44611
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44611
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32563
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:49.278695 32563 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:49.279865 32563 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:49.288975 32574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:49.288985 32572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:49.288997 32563 server_base.cc:1034] running on GCE node
W20260811 15:12:49.289014 32576 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:49.290027 32563 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:49.291365 32563 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:49.292527 32563 hybrid_clock.cc:648] HybridClock initialized: now 1786461169292505 us; error 39 us; skew 500 ppm
Aug 11 15:12:49 dist-test-slave-1vmg krb5kdc[32542](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461169, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:49.295796 32563 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 15:12:49.296818 32563 webserver.cc:460] Webserver started at http://127.0.0.1:45625/ using document root <none> and password file <none>
I20260811 15:12:49.297427 32563 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:49.297499 32563 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:49.297756 32563 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:49.300065 32563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "bbccff29dee44d618219eefb1f4534c1"
format_stamp: "Formatted at 2026-08-11 15:12:49 on dist-test-slave-1vmg"
server_key: "7000183de08398df56d814408e744525"
server_key_iv: "84c27d9b0a9bb7874a3a2e6723d41067"
server_key_version: "encryptionkey@0"
I20260811 15:12:49.300669 32563 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "bbccff29dee44d618219eefb1f4534c1"
format_stamp: "Formatted at 2026-08-11 15:12:49 on dist-test-slave-1vmg"
server_key: "7000183de08398df56d814408e744525"
server_key_iv: "84c27d9b0a9bb7874a3a2e6723d41067"
server_key_version: "encryptionkey@0"
I20260811 15:12:49.304759 32563 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:12:49.307618 32590 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:49.309000 32563 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:12:49.309134 32563 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "bbccff29dee44d618219eefb1f4534c1"
format_stamp: "Formatted at 2026-08-11 15:12:49 on dist-test-slave-1vmg"
server_key: "7000183de08398df56d814408e744525"
server_key_iv: "84c27d9b0a9bb7874a3a2e6723d41067"
server_key_version: "encryptionkey@0"
I20260811 15:12:49.309264 32563 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:49.314400 32563 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:49.317777 32563 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:49.317970 32563 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:49.325850 32563 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44611
I20260811 15:12:49.325953 32675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44611 every 8 connection(s)
I20260811 15:12:49.327188 32563 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:12:49.331241 32676 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:49.333830 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 32563
I20260811 15:12:49.333964 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:12:49.334233 30752 external_mini_cluster.cc:1428] Setting key 5a2a3217caa9b2f57cf23e6aa45e6f0f
I20260811 15:12:49.337675 32676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1: Bootstrap starting.
I20260811 15:12:49.339766 32680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:49.335735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49583 (local address 127.0.0.1:44611)
0811 15:12:49.336227 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:49.336244 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:49.336281 (+ 37us) server_negotiation.cc:407] Connection header received
0811 15:12:49.336929 (+ 648us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:49.336946 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:49.337259 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:49.337589 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:49.338534 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:49.338538 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:49.338549 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:49.338557 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:49.338599 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:49.338617 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:49.338747 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:49.338829 (+ 82us) server_negotiation.cc:299] Negotiation successful
0811 15:12:49.339026 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":164,"threads_started":1}
I20260811 15:12:49.340132 32676 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:49.340931 32676 log.cc:824] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:49.342697 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:12:49.343122 32676 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1: No bootstrap required, opened a new log
I20260811 15:12:49.346655 32680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:49.345110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47471 (local address 127.0.0.1:44611)
0811 15:12:49.345240 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:49.345245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:49.345395 (+ 150us) server_negotiation.cc:407] Connection header received
0811 15:12:49.345484 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:49.345487 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:49.345525 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:49.345606 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:49.346282 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:49.346286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:49.346289 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:49.346302 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:49.346335 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:49.346349 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:49.346443 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:49.346473 (+ 30us) server_negotiation.cc:299] Negotiation successful
0811 15:12:49.346514 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 15:12:49.347853 32676 raft_consensus.cc:348] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbccff29dee44d618219eefb1f4534c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } }
I20260811 15:12:49.348121 32676 raft_consensus.cc:374] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:49.348173 32676 raft_consensus.cc:729] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bbccff29dee44d618219eefb1f4534c1, State: Initialized, Role: FOLLOWER
I20260811 15:12:49.348582 32676 consensus_queue.cc:260] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbccff29dee44d618219eefb1f4534c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } }
I20260811 15:12:49.348697 32676 raft_consensus.cc:388] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:12:49.348742 32676 raft_consensus.cc:482] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:12:49.348819 32676 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:49.349788 32676 raft_consensus.cc:504] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbccff29dee44d618219eefb1f4534c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } }
I20260811 15:12:49.350140 32676 leader_election.cc:302] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bbccff29dee44d618219eefb1f4534c1; no voters:
I20260811 15:12:49.350489 32676 leader_election.cc:288] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:12:49.350778 32691 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:49.351084 32691 raft_consensus.cc:686] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1 [term 1 LEADER]: Becoming Leader. State: Replica: bbccff29dee44d618219eefb1f4534c1, State: Running, Role: LEADER
I20260811 15:12:49.351432 32691 consensus_queue.cc:237] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbccff29dee44d618219eefb1f4534c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } }
I20260811 15:12:49.351889 32676 sys_catalog.cc:564] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:12:49.353029 32693 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bbccff29dee44d618219eefb1f4534c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbccff29dee44d618219eefb1f4534c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } } }
I20260811 15:12:49.353164 32693 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1 [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:49.353500 32699 sys_catalog.cc:455] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bbccff29dee44d618219eefb1f4534c1. Latest consensus state: current_term: 1 leader_uuid: "bbccff29dee44d618219eefb1f4534c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bbccff29dee44d618219eefb1f4534c1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44611 } } }
I20260811 15:12:49.353603 32699 sys_catalog.cc:458] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1 [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:49.353572 32717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:12:49.356734 32717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:12:49.362555 32717 catalog_manager.cc:1347] Generated new cluster ID: d37fca8d92424784a050f5c68568c869
I20260811 15:12:49.362648 32717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:12:49.374995 32717 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:12:49.375368 32717 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:12:49.384634 32717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bbccff29dee44d618219eefb1f4534c1: Generated new TSK 0
I20260811 15:12:49.385457 32717 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:12:49.404780 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 32563
2026-08-11T15:12:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1605 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:12:49 dist-test-slave-1vmg krb5kdc[32738](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:12:49 dist-test-slave-1vmg krb5kdc[32738](info): set up 2 sockets
Aug 11 15:12:49 dist-test-slave-1vmg krb5kdc[32738](info): commencing operation
krb5kdc: starting...
W20260811 15:12:50.602090 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.171s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:12:50 dist-test-slave-1vmg krb5kdc[32738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:12:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:12:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 15:12:50.757975 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36193
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38195
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36193
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:50.897532 32758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:50.897953 32758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:50.898022 32758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:50.902551 32758 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:12:50.902660 32758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:50.902690 32758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:50.902712 32758 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:12:50.902733 32758 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:12:50.909073 32758 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38195
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:50.910701 32758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:50.912144 32758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:50.923285 300 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:50.923290 32767 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:50.923285 303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:50.923625 32758 server_base.cc:1034] running on GCE node
I20260811 15:12:50.924489 32758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:50.925841 32758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:50.927060 32758 hybrid_clock.cc:648] HybridClock initialized: now 1786461170927039 us; error 39 us; skew 500 ppm
Aug 11 15:12:50 dist-test-slave-1vmg krb5kdc[32738](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461170, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:50.930248 32758 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 15:12:50.931313 32758 webserver.cc:460] Webserver started at http://127.0.0.1:41629/ using document root <none> and password file <none>
I20260811 15:12:50.931936 32758 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:50.932010 32758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:50.932240 32758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:50.934516 32758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "88d5bfdce94d45bcbe5f4ab2a1efe86a"
format_stamp: "Formatted at 2026-08-11 15:12:50 on dist-test-slave-1vmg"
server_key: "278d5a6e62372279bfcb29bc0a325422"
server_key_iv: "000e9fbddcf81e4eccefe1dd9a841367"
server_key_version: "encryptionkey@0"
I20260811 15:12:50.935154 32758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "88d5bfdce94d45bcbe5f4ab2a1efe86a"
format_stamp: "Formatted at 2026-08-11 15:12:50 on dist-test-slave-1vmg"
server_key: "278d5a6e62372279bfcb29bc0a325422"
server_key_iv: "000e9fbddcf81e4eccefe1dd9a841367"
server_key_version: "encryptionkey@0"
I20260811 15:12:50.939330 32758 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 15:12:50.941957 314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:50.943347 32758 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:12:50.943480 32758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "88d5bfdce94d45bcbe5f4ab2a1efe86a"
format_stamp: "Formatted at 2026-08-11 15:12:50 on dist-test-slave-1vmg"
server_key: "278d5a6e62372279bfcb29bc0a325422"
server_key_iv: "000e9fbddcf81e4eccefe1dd9a841367"
server_key_version: "encryptionkey@0"
I20260811 15:12:50.943595 32758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:50.959952 32758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:50.963748 32758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:50.963954 32758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:50.971385 32758 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36193
I20260811 15:12:50.971408 410 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36193 every 8 connection(s)
I20260811 15:12:50.972452 32758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:12:50.975668 411 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:50.978008 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 32758
I20260811 15:12:50.978142 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:12:50.978395 30752 external_mini_cluster.cc:1428] Setting key 0da77044481d085395e1039620187e08
I20260811 15:12:50.982487 411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a: Bootstrap starting.
I20260811 15:12:50.985349 411 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:50.986289 411 log.cc:824] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a: Log is configured to *not* fsync() on all Append() calls
Aug 11 15:12:50 dist-test-slave-1vmg krb5kdc[32738](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 15:12:50.988395 411 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a: No bootstrap required, opened a new log
I20260811 15:12:50.993058 411 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d5bfdce94d45bcbe5f4ab2a1efe86a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36193 } }
I20260811 15:12:50.993393 411 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:50.993450 411 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88d5bfdce94d45bcbe5f4ab2a1efe86a, State: Initialized, Role: FOLLOWER
I20260811 15:12:50.993959 411 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d5bfdce94d45bcbe5f4ab2a1efe86a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36193 } }
I20260811 15:12:50.994133 411 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:12:50.994202 411 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:12:50.994288 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:50.980088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:46243 (local address 127.0.0.1:36193)
0811 15:12:50.980560 (+ 472us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:50.980579 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:50.980639 (+ 60us) server_negotiation.cc:407] Connection header received
0811 15:12:50.981462 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:50.981488 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:50.981968 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:50.982351 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:50.983134 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:50.984637 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:50.985523 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:50.986237 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:50.986406 (+ 169us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:50.988914 (+ 2508us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:50.988922 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:50.988941 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:50.988974 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:50.990967 (+ 1993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:50.991475 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:50.991482 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:50.991489 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:50.991650 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:50.991921 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:50.991924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:50.991926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:50.992556 (+ 630us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:50.992803 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:50.992982 (+ 179us) server_negotiation.cc:299] Negotiation successful
0811 15:12:50.993234 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":123,"threads_started":1}
I20260811 15:12:50.994299 411 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:50.995524 411 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d5bfdce94d45bcbe5f4ab2a1efe86a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36193 } }
I20260811 15:12:50.995884 411 leader_election.cc:302] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88d5bfdce94d45bcbe5f4ab2a1efe86a; no voters:
I20260811 15:12:50.996214 411 leader_election.cc:288] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:12:50.996304 420 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:50.996492 420 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a [term 1 LEADER]: Becoming Leader. State: Replica: 88d5bfdce94d45bcbe5f4ab2a1efe86a, State: Running, Role: LEADER
I20260811 15:12:50.996837 420 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d5bfdce94d45bcbe5f4ab2a1efe86a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36193 } }
I20260811 15:12:50.997622 411 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:12:50.998613 422 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88d5bfdce94d45bcbe5f4ab2a1efe86a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d5bfdce94d45bcbe5f4ab2a1efe86a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36193 } } }
I20260811 15:12:50.998729 422 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:50.999046 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:12:50.999239 435 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:12:51.001174 423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88d5bfdce94d45bcbe5f4ab2a1efe86a. Latest consensus state: current_term: 1 leader_uuid: "88d5bfdce94d45bcbe5f4ab2a1efe86a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88d5bfdce94d45bcbe5f4ab2a1efe86a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36193 } } }
I20260811 15:12:51.001278 423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:51.003103 435 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:12:51.009290 435 catalog_manager.cc:1347] Generated new cluster ID: 4a3c2b732de448db8db64068ebe39ef2
I20260811 15:12:51.009383 435 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:12:51.014766 417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:51.001426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:42703 (local address 127.0.0.1:36193)
0811 15:12:51.001533 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:51.001537 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:51.005454 (+ 3917us) server_negotiation.cc:407] Connection header received
0811 15:12:51.005583 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:51.005587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:51.005655 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:51.005752 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:51.006596 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:51.007490 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:51.008709 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:51.009099 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:51.009276 (+ 177us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:51.011288 (+ 2012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:51.011292 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:51.011295 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:51.011323 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:51.012980 (+ 1657us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:51.013516 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:51.013520 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:51.013522 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:51.013607 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:51.013916 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:51.013918 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:51.013920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:51.014118 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:51.014271 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:51.014500 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 15:12:51.014595 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 15:12:51.027223 435 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:12:51.028632 435 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:12:51.037839 435 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88d5bfdce94d45bcbe5f4ab2a1efe86a: Generated new TSK 0
I20260811 15:12:51.038826 435 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:12:51.046506 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 32758
W20260811 15:12:51.065591 32735 connection.cc:535] client connection to 127.0.0.1:36193 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36193): Connection reset by peer (error 104)
2026-08-11T15:12:51Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1649 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:12:51 dist-test-slave-1vmg krb5kdc[462](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:12:51 dist-test-slave-1vmg krb5kdc[462](info): set up 2 sockets
Aug 11 15:12:51 dist-test-slave-1vmg krb5kdc[462](info): commencing operation
krb5kdc: starting...
W20260811 15:12:52.252928 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:12:52 dist-test-slave-1vmg krb5kdc[462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:12:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:12:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 15:12:52.399587 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42725
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36313
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42725
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:52.531788 487 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:52.532191 487 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:52.532240 487 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:52.536788 487 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:12:52.536892 487 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:52.536913 487 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:52.536931 487 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:12:52.536945 487 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:12:52.543292 487 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36313
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.487
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:52.544895 487 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:52.546265 487 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:52.557200 496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:52.557200 498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:52.557257 501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:52.557611 487 server_base.cc:1034] running on GCE node
I20260811 15:12:52.558578 487 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:52.560177 487 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:52.561395 487 hybrid_clock.cc:648] HybridClock initialized: now 1786461172561369 us; error 50 us; skew 500 ppm
Aug 11 15:12:52 dist-test-slave-1vmg krb5kdc[462](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461172, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:52.564723 487 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 15:12:52.565837 487 webserver.cc:460] Webserver started at http://127.0.0.1:45945/ using document root <none> and password file <none>
I20260811 15:12:52.566452 487 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:52.566524 487 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:52.566748 487 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:52.569017 487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "ada17a5930c44799926db2e1578e7e09"
format_stamp: "Formatted at 2026-08-11 15:12:52 on dist-test-slave-1vmg"
server_key: "572d9d80b381e2df993a6a71fc78dcba"
server_key_iv: "637b00b9bf39e1a536267595617c5b9c"
server_key_version: "encryptionkey@0"
I20260811 15:12:52.569602 487 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "ada17a5930c44799926db2e1578e7e09"
format_stamp: "Formatted at 2026-08-11 15:12:52 on dist-test-slave-1vmg"
server_key: "572d9d80b381e2df993a6a71fc78dcba"
server_key_iv: "637b00b9bf39e1a536267595617c5b9c"
server_key_version: "encryptionkey@0"
I20260811 15:12:52.573684 487 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 15:12:52.576344 508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:52.577637 487 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 15:12:52.577808 487 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "ada17a5930c44799926db2e1578e7e09"
format_stamp: "Formatted at 2026-08-11 15:12:52 on dist-test-slave-1vmg"
server_key: "572d9d80b381e2df993a6a71fc78dcba"
server_key_iv: "637b00b9bf39e1a536267595617c5b9c"
server_key_version: "encryptionkey@0"
I20260811 15:12:52.577922 487 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:52.604562 487 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:52.608059 487 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:52.608268 487 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:52.615859 487 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42725
I20260811 15:12:52.615892 594 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42725 every 8 connection(s)
I20260811 15:12:52.616916 487 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:12:52.619719 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 487
I20260811 15:12:52.619882 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:12:52.620157 30752 external_mini_cluster.cc:1428] Setting key 7d07b7aa99abc8f5b310405bd652f690
I20260811 15:12:52.620204 595 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:52.628223 595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09: Bootstrap starting.
I20260811 15:12:52.631016 595 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:52.631027 602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:52.621555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:57183 (local address 127.0.0.1:42725)
0811 15:12:52.622012 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:52.622029 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:52.622066 (+ 37us) server_negotiation.cc:407] Connection header received
0811 15:12:52.623392 (+ 1326us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:52.623434 (+ 42us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:52.624031 (+ 597us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:52.624467 (+ 436us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:52.624949 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:52.626107 (+ 1158us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:52.627476 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:52.628023 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:52.628175 (+ 152us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:52.629186 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:52.629192 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:52.629207 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:52.629628 (+ 421us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:52.629687 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:52.629711 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:52.629819 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:52.630024 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 15:12:52.630183 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":136,"threads_started":1}
I20260811 15:12:52.631850 595 log.cc:824] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:52.634157 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:12:52.634195 595 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09: No bootstrap required, opened a new log
I20260811 15:12:52.639142 595 raft_consensus.cc:348] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada17a5930c44799926db2e1578e7e09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42725 } }
I20260811 15:12:52.639501 595 raft_consensus.cc:374] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:52.639575 595 raft_consensus.cc:729] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ada17a5930c44799926db2e1578e7e09, State: Initialized, Role: FOLLOWER
I20260811 15:12:52.640074 595 consensus_queue.cc:260] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada17a5930c44799926db2e1578e7e09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42725 } }
I20260811 15:12:52.640229 595 raft_consensus.cc:388] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:12:52.640307 595 raft_consensus.cc:482] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:12:52.640422 595 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:52.641237 602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:52.636810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:48337 (local address 127.0.0.1:42725)
0811 15:12:52.636933 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:52.636937 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:52.637110 (+ 173us) server_negotiation.cc:407] Connection header received
0811 15:12:52.637260 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:52.637262 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:52.637305 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:52.637368 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:52.637686 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:52.638207 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:52.639351 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:52.639616 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:52.639707 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:52.640775 (+ 1068us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:52.640780 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:52.640783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:52.640796 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:52.640823 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:52.640834 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:52.640902 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:52.641052 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 15:12:52.641088 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 15:12:52.641640 595 raft_consensus.cc:504] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada17a5930c44799926db2e1578e7e09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42725 } }
I20260811 15:12:52.642060 595 leader_election.cc:302] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ada17a5930c44799926db2e1578e7e09; no voters:
I20260811 15:12:52.642498 595 leader_election.cc:288] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:12:52.642540 612 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:52.642760 612 raft_consensus.cc:686] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09 [term 1 LEADER]: Becoming Leader. State: Replica: ada17a5930c44799926db2e1578e7e09, State: Running, Role: LEADER
I20260811 15:12:52.643131 612 consensus_queue.cc:237] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada17a5930c44799926db2e1578e7e09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42725 } }
I20260811 15:12:52.643899 595 sys_catalog.cc:564] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:12:52.644942 613 sys_catalog.cc:455] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ada17a5930c44799926db2e1578e7e09. Latest consensus state: current_term: 1 leader_uuid: "ada17a5930c44799926db2e1578e7e09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada17a5930c44799926db2e1578e7e09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42725 } } }
I20260811 15:12:52.644932 616 sys_catalog.cc:455] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ada17a5930c44799926db2e1578e7e09" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ada17a5930c44799926db2e1578e7e09" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42725 } } }
I20260811 15:12:52.645102 613 sys_catalog.cc:458] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09 [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:52.645109 616 sys_catalog.cc:458] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09 [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:52.645532 627 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:12:52.649348 627 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:12:52.655242 627 catalog_manager.cc:1347] Generated new cluster ID: 9c6b098fe3614bdead45b26e841b4b8f
I20260811 15:12:52.655336 627 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:12:52.702700 627 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:12:52.703819 627 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:12:52.710319 627 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ada17a5930c44799926db2e1578e7e09: Generated new TSK 0
I20260811 15:12:52.711114 627 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:12:52.716450 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 487
W20260811 15:12:52.733397 460 connection.cc:535] client connection to 127.0.0.1:42725 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42725): Connection reset by peer (error 104)
2026-08-11T15:12:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1667 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:12:52 dist-test-slave-1vmg krb5kdc[648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:12:52 dist-test-slave-1vmg krb5kdc[648](info): set up 2 sockets
Aug 11 15:12:52 dist-test-slave-1vmg krb5kdc[648](info): commencing operation
krb5kdc: starting...
W20260811 15:12:53.922520 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.174s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:12:54 dist-test-slave-1vmg krb5kdc[648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:12:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:12:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 15:12:54.072587 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40493
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:54.193944 672 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:54.194291 672 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:54.194353 672 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:54.198510 672 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:12:54.198606 672 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:54.198633 672 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:54.198655 672 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:12:54.198675 672 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:12:54.204728 672 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40493
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.672
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:54.206311 672 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:54.207509 672 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:54.216665 685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:54.216773 682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:54.216689 688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:54.217056 672 server_base.cc:1034] running on GCE node
I20260811 15:12:54.218015 672 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:54.219708 672 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:54.220947 672 hybrid_clock.cc:648] HybridClock initialized: now 1786461174220903 us; error 63 us; skew 500 ppm
Aug 11 15:12:54 dist-test-slave-1vmg krb5kdc[648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461174, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:54.224190 672 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 15:12:54.225256 672 webserver.cc:460] Webserver started at http://127.0.0.1:46545/ using document root <none> and password file <none>
I20260811 15:12:54.225855 672 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:54.225926 672 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:54.226161 672 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:54.228425 672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "20d15cfbfd744c6f8363b40640c5196a"
format_stamp: "Formatted at 2026-08-11 15:12:54 on dist-test-slave-1vmg"
server_key: "2c8ecde4c1057ca99b99dc2f8c866769"
server_key_iv: "bcede04c9b180bc5225d4045f3476b14"
server_key_version: "encryptionkey@0"
I20260811 15:12:54.229106 672 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "20d15cfbfd744c6f8363b40640c5196a"
format_stamp: "Formatted at 2026-08-11 15:12:54 on dist-test-slave-1vmg"
server_key: "2c8ecde4c1057ca99b99dc2f8c866769"
server_key_iv: "bcede04c9b180bc5225d4045f3476b14"
server_key_version: "encryptionkey@0"
I20260811 15:12:54.233246 672 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:12:54.235849 699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:54.237138 672 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:12:54.237267 672 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "20d15cfbfd744c6f8363b40640c5196a"
format_stamp: "Formatted at 2026-08-11 15:12:54 on dist-test-slave-1vmg"
server_key: "2c8ecde4c1057ca99b99dc2f8c866769"
server_key_iv: "bcede04c9b180bc5225d4045f3476b14"
server_key_version: "encryptionkey@0"
I20260811 15:12:54.237377 672 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:54.242383 672 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:54.245623 672 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:54.245811 672 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:54.253043 672 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40493
I20260811 15:12:54.253075 796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40493 every 8 connection(s)
I20260811 15:12:54.254040 672 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:12:54.257091 798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:54.260317 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 672
I20260811 15:12:54.260480 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:12:54.260740 30752 external_mini_cluster.cc:1428] Setting key 06a4e7ceeb2f5683b1b3f605a6ac4d43
I20260811 15:12:54.263700 798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a: Bootstrap starting.
I20260811 15:12:54.266501 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:54.262236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:59529 (local address 127.0.0.1:40493)
0811 15:12:54.262690 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:54.262712 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:54.262758 (+ 46us) server_negotiation.cc:407] Connection header received
0811 15:12:54.263652 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:54.263678 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:54.264053 (+ 375us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:54.264440 (+ 387us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:54.265085 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:54.265092 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:54.265107 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:54.265118 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:54.265176 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:54.265219 (+ 43us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:54.265356 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:54.265423 (+ 67us) server_negotiation.cc:299] Negotiation successful
0811 15:12:54.265580 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":296,"thread_start_us":129,"threads_started":1}
I20260811 15:12:54.266546 798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:54.267398 798 log.cc:824] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:54.269341 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:12:54.269454 798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a: No bootstrap required, opened a new log
I20260811 15:12:54.273296 802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:54.271764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:44161 (local address 127.0.0.1:40493)
0811 15:12:54.271888 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:54.271892 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:54.272028 (+ 136us) server_negotiation.cc:407] Connection header received
0811 15:12:54.272148 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:54.272150 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:54.272183 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:54.272247 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:54.272942 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:54.272946 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:54.272947 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:54.272958 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:54.272984 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:54.272994 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:54.273063 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:54.273150 (+ 87us) server_negotiation.cc:299] Negotiation successful
0811 15:12:54.273183 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 15:12:54.274053 798 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20d15cfbfd744c6f8363b40640c5196a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40493 } }
I20260811 15:12:54.274299 798 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:54.274366 798 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20d15cfbfd744c6f8363b40640c5196a, State: Initialized, Role: FOLLOWER
I20260811 15:12:54.274827 798 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20d15cfbfd744c6f8363b40640c5196a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40493 } }
I20260811 15:12:54.275007 798 raft_consensus.cc:388] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:12:54.275076 798 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:12:54.275182 798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:54.276153 798 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20d15cfbfd744c6f8363b40640c5196a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40493 } }
I20260811 15:12:54.276506 798 leader_election.cc:302] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 20d15cfbfd744c6f8363b40640c5196a; no voters:
I20260811 15:12:54.276827 798 leader_election.cc:288] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:12:54.276923 814 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:54.277132 814 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a [term 1 LEADER]: Becoming Leader. State: Replica: 20d15cfbfd744c6f8363b40640c5196a, State: Running, Role: LEADER
I20260811 15:12:54.277457 814 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20d15cfbfd744c6f8363b40640c5196a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40493 } }
I20260811 15:12:54.278113 798 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:12:54.279342 818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20d15cfbfd744c6f8363b40640c5196a. Latest consensus state: current_term: 1 leader_uuid: "20d15cfbfd744c6f8363b40640c5196a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20d15cfbfd744c6f8363b40640c5196a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40493 } } }
I20260811 15:12:54.279470 818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:54.279822 817 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "20d15cfbfd744c6f8363b40640c5196a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "20d15cfbfd744c6f8363b40640c5196a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40493 } } }
I20260811 15:12:54.279920 817 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:54.280603 829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:12:54.283514 829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:12:54.289673 829 catalog_manager.cc:1347] Generated new cluster ID: 40cac7535e3e4b28ac8337582ec595e2
I20260811 15:12:54.289765 829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:12:54.304102 829 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:12:54.304544 829 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:12:54.311002 829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 20d15cfbfd744c6f8363b40640c5196a: Generated new TSK 0
I20260811 15:12:54.312011 829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:12:54.320071 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 672
2026-08-11T15:12:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1601 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:12:54 dist-test-slave-1vmg krb5kdc[852](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:12:54 dist-test-slave-1vmg krb5kdc[852](info): set up 2 sockets
Aug 11 15:12:54 dist-test-slave-1vmg krb5kdc[852](info): commencing operation
krb5kdc: starting...
W20260811 15:12:55.521612 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:12:55 dist-test-slave-1vmg krb5kdc[852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:12:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:12:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 15:12:55.658967 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43831
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:55.778223 868 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:55.778549 868 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:55.778594 868 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:55.782788 868 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:12:55.782891 868 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:55.782941 868 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:55.782960 868 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:12:55.782979 868 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:12:55.788771 868 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43831
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.868
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:55.789979 868 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:55.791072 868 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 15:12:55.800031 868 server_base.cc:1034] running on GCE node
W20260811 15:12:55.799979 879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:55.799979 880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:55.800012 883 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:55.800815 868 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:55.802069 868 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:55.803246 868 hybrid_clock.cc:648] HybridClock initialized: now 1786461175803218 us; error 39 us; skew 500 ppm
Aug 11 15:12:55 dist-test-slave-1vmg krb5kdc[852](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461175, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:55.813270 868 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 15:12:55.814368 868 webserver.cc:460] Webserver started at http://127.0.0.1:35337/ using document root <none> and password file <none>
I20260811 15:12:55.815390 868 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:55.815562 868 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:55.815873 868 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:55.818763 868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "06bafb2c2c0d44a28d6a24f47c4f768f"
format_stamp: "Formatted at 2026-08-11 15:12:55 on dist-test-slave-1vmg"
server_key: "67f45e478cad35ce21654777ec6d77a5"
server_key_iv: "787fd59ed039014ececa2eca71cb2db2"
server_key_version: "encryptionkey@0"
I20260811 15:12:55.819559 868 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "06bafb2c2c0d44a28d6a24f47c4f768f"
format_stamp: "Formatted at 2026-08-11 15:12:55 on dist-test-slave-1vmg"
server_key: "67f45e478cad35ce21654777ec6d77a5"
server_key_iv: "787fd59ed039014ececa2eca71cb2db2"
server_key_version: "encryptionkey@0"
I20260811 15:12:55.824729 868 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260811 15:12:55.827769 895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:55.829175 868 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 15:12:55.829308 868 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "06bafb2c2c0d44a28d6a24f47c4f768f"
format_stamp: "Formatted at 2026-08-11 15:12:55 on dist-test-slave-1vmg"
server_key: "67f45e478cad35ce21654777ec6d77a5"
server_key_iv: "787fd59ed039014ececa2eca71cb2db2"
server_key_version: "encryptionkey@0"
I20260811 15:12:55.829440 868 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:55.844475 868 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:55.847889 868 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:55.848090 868 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:55.855475 868 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43831
I20260811 15:12:55.855505 985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43831 every 8 connection(s)
I20260811 15:12:55.856537 868 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:12:55.858296 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 868
I20260811 15:12:55.858434 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:12:55.858700 30752 external_mini_cluster.cc:1428] Setting key 4dde746da6871fe40b4f6d5dc6475d8f
I20260811 15:12:55.859803 989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:55.865856 989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f: Bootstrap starting.
Aug 11 15:12:55 dist-test-slave-1vmg krb5kdc[852](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 15:12:55.868462 989 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:55.869231 989 log.cc:824] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:55.871467 989 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f: No bootstrap required, opened a new log
I20260811 15:12:55.873339 993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:55.860212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:43151 (local address 127.0.0.1:43831)
0811 15:12:55.860695 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:55.860712 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:55.860749 (+ 37us) server_negotiation.cc:407] Connection header received
0811 15:12:55.861426 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:55.861443 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:55.861799 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:55.862120 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:55.862970 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:55.864007 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:55.864854 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:55.865344 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:55.865492 (+ 148us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:55.868227 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:55.868233 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:55.868249 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:55.868281 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:55.870129 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:55.870656 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:55.870661 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:55.870667 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:55.870781 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:55.871155 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:55.871157 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:55.871159 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:55.871607 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:55.871821 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:55.872134 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 15:12:55.872363 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":158,"threads_started":1}
I20260811 15:12:55.874768 989 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06bafb2c2c0d44a28d6a24f47c4f768f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43831 } }
I20260811 15:12:55.875078 989 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:55.875140 989 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06bafb2c2c0d44a28d6a24f47c4f768f, State: Initialized, Role: FOLLOWER
I20260811 15:12:55.875582 989 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06bafb2c2c0d44a28d6a24f47c4f768f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43831 } }
I20260811 15:12:55.875725 989 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:12:55.875792 989 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:12:55.875895 989 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:55.876574 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:12:55.877064 989 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06bafb2c2c0d44a28d6a24f47c4f768f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43831 } }
I20260811 15:12:55.877411 989 leader_election.cc:302] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06bafb2c2c0d44a28d6a24f47c4f768f; no voters:
I20260811 15:12:55.877727 989 leader_election.cc:288] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:12:55.877826 995 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:55.878059 995 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f [term 1 LEADER]: Becoming Leader. State: Replica: 06bafb2c2c0d44a28d6a24f47c4f768f, State: Running, Role: LEADER
I20260811 15:12:55.878429 995 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06bafb2c2c0d44a28d6a24f47c4f768f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43831 } }
I20260811 15:12:55.880306 989 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:12:55.880336 1004 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06bafb2c2c0d44a28d6a24f47c4f768f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06bafb2c2c0d44a28d6a24f47c4f768f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43831 } } }
I20260811 15:12:55.880455 1004 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:55.880770 1003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06bafb2c2c0d44a28d6a24f47c4f768f. Latest consensus state: current_term: 1 leader_uuid: "06bafb2c2c0d44a28d6a24f47c4f768f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06bafb2c2c0d44a28d6a24f47c4f768f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43831 } } }
I20260811 15:12:55.880856 1003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:55.881289 1009 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:12:55.885074 1009 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:12:55.891017 993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:55.879397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:35451 (local address 127.0.0.1:43831)
0811 15:12:55.879572 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:55.879576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:55.879817 (+ 241us) server_negotiation.cc:407] Connection header received
0811 15:12:55.880767 (+ 950us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:55.880770 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:55.880825 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:55.880879 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:55.882407 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:55.883295 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:55.884539 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:55.884888 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:55.884996 (+ 108us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:55.886731 (+ 1735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:55.886735 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:55.886739 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:12:55.886767 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:55.888580 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:55.889251 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:55.889255 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:55.889257 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:55.889350 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:12:55.889765 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:12:55.889769 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:12:55.889771 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:12:55.890014 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:55.890177 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:55.890516 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 15:12:55.890626 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 15:12:55.891337 1009 catalog_manager.cc:1347] Generated new cluster ID: d7eaab6ee5764182a37fc9dd6236e848
I20260811 15:12:55.891456 1009 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:12:55.903323 1009 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:12:55.905247 1009 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:12:55.914409 1009 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06bafb2c2c0d44a28d6a24f47c4f768f: Generated new TSK 0
I20260811 15:12:55.915371 1009 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:12:55.919880 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 868
W20260811 15:12:55.947300 849 connection.cc:535] client connection to 127.0.0.1:43831 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43831): Connection reset by peer (error 104)
2026-08-11T15:12:55Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1612 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:12:56 dist-test-slave-1vmg krb5kdc[1036](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:12:56 dist-test-slave-1vmg krb5kdc[1036](info): set up 2 sockets
Aug 11 15:12:56 dist-test-slave-1vmg krb5kdc[1036](info): commencing operation
krb5kdc: starting...
W20260811 15:12:57.134318 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:12:57 dist-test-slave-1vmg krb5kdc[1036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:12:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:12:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 15:12:57.277626 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41703
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33737
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:57.428383 1058 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:57.428699 1058 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:57.428761 1058 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:57.432984 1058 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:12:57.433079 1058 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:57.433127 1058 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:57.433159 1058 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:12:57.433192 1058 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:12:57.439004 1058 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33737
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41703
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1058
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:57.440305 1058 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:57.441413 1058 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:57.451390 1071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:57.451402 1065 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:57.451643 1058 server_base.cc:1034] running on GCE node
W20260811 15:12:57.451402 1066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:57.452407 1058 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:57.453827 1058 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:57.455003 1058 hybrid_clock.cc:648] HybridClock initialized: now 1786461177454982 us; error 33 us; skew 500 ppm
Aug 11 15:12:57 dist-test-slave-1vmg krb5kdc[1036](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461177, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:57.458385 1058 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 15:12:57.459890 1058 webserver.cc:460] Webserver started at http://127.0.0.1:34371/ using document root <none> and password file <none>
I20260811 15:12:57.460637 1058 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:57.460716 1058 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:57.460971 1058 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:57.463747 1058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "f4467ad145fe4212931a0c6a9ee9c7fd"
format_stamp: "Formatted at 2026-08-11 15:12:57 on dist-test-slave-1vmg"
server_key: "9f69d878f412c9f4b90954fd8d4830f6"
server_key_iv: "b67ddebaf4946deb8f0b0f1ee6a7008a"
server_key_version: "encryptionkey@0"
I20260811 15:12:57.464416 1058 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "f4467ad145fe4212931a0c6a9ee9c7fd"
format_stamp: "Formatted at 2026-08-11 15:12:57 on dist-test-slave-1vmg"
server_key: "9f69d878f412c9f4b90954fd8d4830f6"
server_key_iv: "b67ddebaf4946deb8f0b0f1ee6a7008a"
server_key_version: "encryptionkey@0"
I20260811 15:12:57.469507 1058 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 15:12:57.472954 1086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:57.474506 1058 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:12:57.474658 1058 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "f4467ad145fe4212931a0c6a9ee9c7fd"
format_stamp: "Formatted at 2026-08-11 15:12:57 on dist-test-slave-1vmg"
server_key: "9f69d878f412c9f4b90954fd8d4830f6"
server_key_iv: "b67ddebaf4946deb8f0b0f1ee6a7008a"
server_key_version: "encryptionkey@0"
I20260811 15:12:57.474814 1058 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:57.494151 1058 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:57.497330 1058 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:57.497537 1058 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:57.504812 1058 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41703
I20260811 15:12:57.504832 1157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41703 every 8 connection(s)
I20260811 15:12:57.505872 1058 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:12:57.507759 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 1058
I20260811 15:12:57.507911 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:12:57.508220 30752 external_mini_cluster.cc:1428] Setting key b543f252de38e3de93237ed7a7621adc
I20260811 15:12:57.509464 1159 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:57.516193 1159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd: Bootstrap starting.
I20260811 15:12:57.517437 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:57.509624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:52199 (local address 127.0.0.1:41703)
0811 15:12:57.510047 (+ 423us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:57.510064 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:57.510099 (+ 35us) server_negotiation.cc:407] Connection header received
0811 15:12:57.510776 (+ 677us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:57.510793 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:57.511208 (+ 415us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:57.511492 (+ 284us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:57.511829 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:57.512834 (+ 1005us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:57.513991 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:57.514580 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:57.514699 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:57.515648 (+ 949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:57.515653 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:57.515667 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:57.516084 (+ 417us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:57.516142 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:57.516165 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:57.516267 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:57.516524 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 15:12:57.516681 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":112,"threads_started":1}
I20260811 15:12:57.518697 1159 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:57.519627 1159 log.cc:824] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:57.520684 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:12:57.521795 1159 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd: No bootstrap required, opened a new log
I20260811 15:12:57.525560 1159 raft_consensus.cc:348] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4467ad145fe4212931a0c6a9ee9c7fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41703 } }
I20260811 15:12:57.525869 1159 raft_consensus.cc:374] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:57.525914 1159 raft_consensus.cc:729] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4467ad145fe4212931a0c6a9ee9c7fd, State: Initialized, Role: FOLLOWER
I20260811 15:12:57.526363 1159 consensus_queue.cc:260] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4467ad145fe4212931a0c6a9ee9c7fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41703 } }
I20260811 15:12:57.526501 1159 raft_consensus.cc:388] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:12:57.526569 1159 raft_consensus.cc:482] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:12:57.526676 1159 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:57.527501 1163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:57.522676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:49053 (local address 127.0.0.1:41703)
0811 15:12:57.522791 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:57.522795 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:57.523065 (+ 270us) server_negotiation.cc:407] Connection header received
0811 15:12:57.523162 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:57.523165 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:57.523258 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:57.523372 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:57.523733 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:57.524603 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:57.525697 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:12:57.525940 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:12:57.526031 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:12:57.527053 (+ 1022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:57.527057 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:57.527060 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:57.527075 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:57.527100 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:57.527110 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:57.527186 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:57.527327 (+ 141us) server_negotiation.cc:299] Negotiation successful
0811 15:12:57.527367 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 15:12:57.527817 1159 raft_consensus.cc:504] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4467ad145fe4212931a0c6a9ee9c7fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41703 } }
I20260811 15:12:57.528219 1159 leader_election.cc:302] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f4467ad145fe4212931a0c6a9ee9c7fd; no voters:
I20260811 15:12:57.528596 1159 leader_election.cc:288] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:12:57.528726 1177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:57.528936 1177 raft_consensus.cc:686] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd [term 1 LEADER]: Becoming Leader. State: Replica: f4467ad145fe4212931a0c6a9ee9c7fd, State: Running, Role: LEADER
I20260811 15:12:57.529296 1177 consensus_queue.cc:237] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4467ad145fe4212931a0c6a9ee9c7fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41703 } }
I20260811 15:12:57.529908 1159 sys_catalog.cc:564] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:12:57.531009 1180 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader f4467ad145fe4212931a0c6a9ee9c7fd. Latest consensus state: current_term: 1 leader_uuid: "f4467ad145fe4212931a0c6a9ee9c7fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4467ad145fe4212931a0c6a9ee9c7fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41703 } } }
I20260811 15:12:57.531142 1180 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:57.531476 1179 sys_catalog.cc:455] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f4467ad145fe4212931a0c6a9ee9c7fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f4467ad145fe4212931a0c6a9ee9c7fd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41703 } } }
I20260811 15:12:57.531625 1179 sys_catalog.cc:458] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:57.531527 1190 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:12:57.535202 1190 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:12:57.541291 1190 catalog_manager.cc:1347] Generated new cluster ID: 2c745c1a551a4fbbbef14a284adbc89a
I20260811 15:12:57.541386 1190 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:12:57.552192 1190 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:12:57.553611 1190 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:12:57.571446 1190 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f4467ad145fe4212931a0c6a9ee9c7fd: Generated new TSK 0
I20260811 15:12:57.572456 1190 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:12:57.577955 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 1058
W20260811 15:12:57.590891 1034 connection.cc:535] client connection to 127.0.0.1:41703 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41703): Connection reset by peer (error 104)
2026-08-11T15:12:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1643 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:12:57 dist-test-slave-1vmg krb5kdc[1210](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:12:57 dist-test-slave-1vmg krb5kdc[1210](info): set up 2 sockets
Aug 11 15:12:57 dist-test-slave-1vmg krb5kdc[1210](info): commencing operation
krb5kdc: starting...
W20260811 15:12:58.754019 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.003s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:12:58 dist-test-slave-1vmg krb5kdc[1210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:12:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:12:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 15:12:58.898307 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37403
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37403
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:12:59.038404 1231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:12:59.038787 1231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:12:59.038836 1231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:12:59.043203 1231 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:12:59.043300 1231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:12:59.043319 1231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:12:59.043336 1231 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:12:59.043361 1231 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:12:59.049571 1231 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37403
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37403
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:12:59.051158 1231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:12:59.052491 1231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:12:59.063562 1245 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:59.063555 1242 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:12:59.063555 1241 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:12:59.064133 1231 server_base.cc:1034] running on GCE node
I20260811 15:12:59.064857 1231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:12:59.066190 1231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:12:59.067327 1231 hybrid_clock.cc:648] HybridClock initialized: now 1786461179067313 us; error 40 us; skew 500 ppm
Aug 11 15:12:59 dist-test-slave-1vmg krb5kdc[1210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461179, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:12:59.070299 1231 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 15:12:59.071699 1231 webserver.cc:460] Webserver started at http://127.0.0.1:38081/ using document root <none> and password file <none>
I20260811 15:12:59.072353 1231 fs_manager.cc:359] Metadata directory not provided
I20260811 15:12:59.072422 1231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:12:59.072670 1231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:12:59.075037 1231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "c9db6957f8f4444db0b9a15bd97916eb"
format_stamp: "Formatted at 2026-08-11 15:12:59 on dist-test-slave-1vmg"
server_key: "be9c851f69e40e9ca3fc824c5b487d1e"
server_key_iv: "dead827fe0e78e104100807bd131a474"
server_key_version: "encryptionkey@0"
I20260811 15:12:59.075644 1231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "c9db6957f8f4444db0b9a15bd97916eb"
format_stamp: "Formatted at 2026-08-11 15:12:59 on dist-test-slave-1vmg"
server_key: "be9c851f69e40e9ca3fc824c5b487d1e"
server_key_iv: "dead827fe0e78e104100807bd131a474"
server_key_version: "encryptionkey@0"
I20260811 15:12:59.079941 1231 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:12:59.082621 1255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:12:59.084007 1231 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 15:12:59.084146 1231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "c9db6957f8f4444db0b9a15bd97916eb"
format_stamp: "Formatted at 2026-08-11 15:12:59 on dist-test-slave-1vmg"
server_key: "be9c851f69e40e9ca3fc824c5b487d1e"
server_key_iv: "dead827fe0e78e104100807bd131a474"
server_key_version: "encryptionkey@0"
I20260811 15:12:59.084260 1231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:12:59.089316 1231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:12:59.092381 1231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:12:59.092583 1231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:12:59.100042 1231 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37403
I20260811 15:12:59.100051 1345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37403 every 8 connection(s)
I20260811 15:12:59.101102 1231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:12:59.104243 1347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:12:59.108160 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 1231
I20260811 15:12:59.108321 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:12:59.108630 30752 external_mini_cluster.cc:1428] Setting key 94b6af3543ce24b689d6a86671625734
I20260811 15:12:59.110790 1347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb: Bootstrap starting.
I20260811 15:12:59.113404 1347 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb: Neither blocks nor log segments found. Creating new log.
I20260811 15:12:59.114094 1352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:59.110084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:36991 (local address 127.0.0.1:37403)
0811 15:12:59.110541 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:59.110557 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:59.110597 (+ 40us) server_negotiation.cc:407] Connection header received
0811 15:12:59.111320 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:59.111341 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:59.111660 (+ 319us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:59.111958 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:59.112814 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:59.112819 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:59.112831 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:59.112842 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:59.112902 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:59.112921 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:59.113008 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:59.113121 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 15:12:59.113283 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293,"thread_start_us":122,"threads_started":1}
I20260811 15:12:59.114168 1347 log.cc:824] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb: Log is configured to *not* fsync() on all Append() calls
I20260811 15:12:59.116292 1347 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb: No bootstrap required, opened a new log
I20260811 15:12:59.116871 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:12:59.120793 1352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:12:59.119249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.118:40371 (local address 127.0.0.1:37403)
0811 15:12:59.119373 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 15:12:59.119378 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:12:59.119573 (+ 195us) server_negotiation.cc:407] Connection header received
0811 15:12:59.119658 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:12:59.119661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:12:59.119704 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:12:59.119809 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:12:59.120432 (+ 623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:12:59.120435 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:12:59.120438 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:12:59.120450 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:12:59.120479 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:12:59.120491 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:12:59.120581 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 15:12:59.120628 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 15:12:59.120667 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 15:12:59.120918 1347 raft_consensus.cc:348] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9db6957f8f4444db0b9a15bd97916eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37403 } }
I20260811 15:12:59.121194 1347 raft_consensus.cc:374] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:12:59.121253 1347 raft_consensus.cc:729] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9db6957f8f4444db0b9a15bd97916eb, State: Initialized, Role: FOLLOWER
I20260811 15:12:59.121748 1347 consensus_queue.cc:260] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9db6957f8f4444db0b9a15bd97916eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37403 } }
I20260811 15:12:59.121894 1347 raft_consensus.cc:388] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:12:59.121958 1347 raft_consensus.cc:482] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:12:59.122069 1347 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:12:59.123086 1347 raft_consensus.cc:504] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9db6957f8f4444db0b9a15bd97916eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37403 } }
I20260811 15:12:59.123483 1347 leader_election.cc:302] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c9db6957f8f4444db0b9a15bd97916eb; no voters:
I20260811 15:12:59.123816 1347 leader_election.cc:288] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:12:59.123962 1364 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:12:59.124225 1364 raft_consensus.cc:686] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb [term 1 LEADER]: Becoming Leader. State: Replica: c9db6957f8f4444db0b9a15bd97916eb, State: Running, Role: LEADER
I20260811 15:12:59.124600 1364 consensus_queue.cc:237] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9db6957f8f4444db0b9a15bd97916eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37403 } }
I20260811 15:12:59.125104 1347 sys_catalog.cc:564] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:12:59.126407 1366 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb [sys.catalog]: SysCatalogTable state changed. Reason: New leader c9db6957f8f4444db0b9a15bd97916eb. Latest consensus state: current_term: 1 leader_uuid: "c9db6957f8f4444db0b9a15bd97916eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9db6957f8f4444db0b9a15bd97916eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37403 } } }
I20260811 15:12:59.126456 1365 sys_catalog.cc:455] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c9db6957f8f4444db0b9a15bd97916eb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c9db6957f8f4444db0b9a15bd97916eb" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37403 } } }
I20260811 15:12:59.126595 1365 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:59.126849 1366 sys_catalog.cc:458] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb [sys.catalog]: This master's current role is: LEADER
I20260811 15:12:59.127182 1381 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:12:59.130122 1381 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:12:59.136153 1381 catalog_manager.cc:1347] Generated new cluster ID: 98523de4dd9c4c4eb6428cf42cd930d3
I20260811 15:12:59.136246 1381 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:12:59.153759 1381 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:12:59.154124 1381 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:12:59.158830 1381 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c9db6957f8f4444db0b9a15bd97916eb: Generated new TSK 0
I20260811 15:12:59.160146 1381 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:12:59.171141 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 1231
2026-08-11T15:12:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1610 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:12:59 dist-test-slave-1vmg krb5kdc[1395](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:12:59 dist-test-slave-1vmg krb5kdc[1395](info): set up 2 sockets
Aug 11 15:12:59 dist-test-slave-1vmg krb5kdc[1395](info): commencing operation
krb5kdc: starting...
W20260811 15:13:00.375260 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.159s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:13:00 dist-test-slave-1vmg krb5kdc[1395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:13:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:13:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:13:00.521890 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:44809
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:35859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:44809
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:13:00.650092 1420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:13:00.650421 1420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:13:00.650485 1420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:13:00.654683 1420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:13:00.654785 1420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:13:00.654814 1420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:13:00.654834 1420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:13:00.654855 1420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:13:00.660650 1420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:35859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:44809
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:44809
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:13:00.661928 1420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:13:00.663125 1420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 15:13:00.672339 1420 server_base.cc:1034] running on GCE node
W20260811 15:13:00.672235 1427 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:00.672228 1428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:00.672241 1430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:13:00.673120 1420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:13:00.674302 1420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:13:00.675480 1420 hybrid_clock.cc:648] HybridClock initialized: now 1786461180675461 us; error 36 us; skew 500 ppm
Aug 11 15:13:00 dist-test-slave-1vmg krb5kdc[1395](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461180, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:13:00.678905 1420 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:13:00.680362 1420 webserver.cc:460] Webserver started at http://127.30.8.62:33037/ using document root <none> and password file <none>
I20260811 15:13:00.681131 1420 fs_manager.cc:359] Metadata directory not provided
I20260811 15:13:00.681209 1420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:13:00.681485 1420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:13:00.684270 1420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "7b72c81bc502411ba82f420acdbed297"
format_stamp: "Formatted at 2026-08-11 15:13:00 on dist-test-slave-1vmg"
server_key: "09c7e826f57fa0490c2a6638d9960d75"
server_key_iv: "712d13d38d6e01054b9808e61304705a"
server_key_version: "encryptionkey@0"
I20260811 15:13:00.684960 1420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "7b72c81bc502411ba82f420acdbed297"
format_stamp: "Formatted at 2026-08-11 15:13:00 on dist-test-slave-1vmg"
server_key: "09c7e826f57fa0490c2a6638d9960d75"
server_key_iv: "712d13d38d6e01054b9808e61304705a"
server_key_version: "encryptionkey@0"
I20260811 15:13:00.690152 1420 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 15:13:00.693218 1440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:13:00.694581 1420 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:13:00.694723 1420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "7b72c81bc502411ba82f420acdbed297"
format_stamp: "Formatted at 2026-08-11 15:13:00 on dist-test-slave-1vmg"
server_key: "09c7e826f57fa0490c2a6638d9960d75"
server_key_iv: "712d13d38d6e01054b9808e61304705a"
server_key_version: "encryptionkey@0"
I20260811 15:13:00.694854 1420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:13:00.712935 1420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:13:00.716244 1420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:13:00.716440 1420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:13:00.724114 1420 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:44809
I20260811 15:13:00.724121 1537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:44809 every 8 connection(s)
I20260811 15:13:00.725183 1420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:13:00.728229 1540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:13:00.731673 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 1420
I20260811 15:13:00.731832 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:13:00.732136 30752 external_mini_cluster.cc:1428] Setting key 23edc20cdf558a6326004c12f3bc275f
I20260811 15:13:00.734782 1540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297: Bootstrap starting.
I20260811 15:13:00.737138 1540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297: Neither blocks nor log segments found. Creating new log.
I20260811 15:13:00.737903 1540 log.cc:824] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297: Log is configured to *not* fsync() on all Append() calls
I20260811 15:13:00.740037 1540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297: No bootstrap required, opened a new log
Aug 11 15:13:00 dist-test-slave-1vmg krb5kdc[1395](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:13:00.744148 1540 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b72c81bc502411ba82f420acdbed297" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44809 } }
I20260811 15:13:00.744400 1540 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:13:00.744433 1540 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b72c81bc502411ba82f420acdbed297, State: Initialized, Role: FOLLOWER
I20260811 15:13:00.744884 1540 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b72c81bc502411ba82f420acdbed297" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44809 } }
I20260811 15:13:00.745008 1540 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:13:00.745071 1540 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:13:00.745173 1540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:13:00.746147 1540 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b72c81bc502411ba82f420acdbed297" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44809 } }
I20260811 15:13:00.746536 1540 leader_election.cc:302] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7b72c81bc502411ba82f420acdbed297; no voters:
I20260811 15:13:00.746907 1540 leader_election.cc:288] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:13:00.747052 1545 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:13:00.747296 1545 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297 [term 1 LEADER]: Becoming Leader. State: Replica: 7b72c81bc502411ba82f420acdbed297, State: Running, Role: LEADER
I20260811 15:13:00.747653 1545 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b72c81bc502411ba82f420acdbed297" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44809 } }
I20260811 15:13:00.747867 1543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:00.733713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52243 (local address 127.30.8.62:44809)
0811 15:13:00.734205 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:00.734221 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:00.734260 (+ 39us) server_negotiation.cc:407] Connection header received
0811 15:13:00.734965 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:00.734986 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:00.735354 (+ 368us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:00.735671 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:00.736533 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:00.737581 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:00.738449 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:00.738968 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:00.739133 (+ 165us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:00.742086 (+ 2953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:00.742104 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:00.742135 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:00.742212 (+ 77us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:00.744398 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:00.744976 (+ 578us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:00.744983 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:00.744990 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:00.745143 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:00.745489 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:00.745492 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:00.745494 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:00.745936 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:00.746147 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:00.746493 (+ 346us) server_negotiation.cc:299] Negotiation successful
0811 15:13:00.746762 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":147,"threads_started":1}
I20260811 15:13:00.748330 1540 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:13:00.748875 1545 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7b72c81bc502411ba82f420acdbed297. Latest consensus state: current_term: 1 leader_uuid: "7b72c81bc502411ba82f420acdbed297" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b72c81bc502411ba82f420acdbed297" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44809 } } }
I20260811 15:13:00.748996 1545 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297 [sys.catalog]: This master's current role is: LEADER
I20260811 15:13:00.749432 1547 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7b72c81bc502411ba82f420acdbed297" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7b72c81bc502411ba82f420acdbed297" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44809 } } }
I20260811 15:13:00.749533 1547 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297 [sys.catalog]: This master's current role is: LEADER
I20260811 15:13:00.751358 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260811 15:13:00.753255 1567 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:13:00.753324 1567 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:13:00.753407 1552 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:13:00.754094 1552 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:13:00.759819 1552 catalog_manager.cc:1347] Generated new cluster ID: 0727a520f82841059c49df39a0549570
I20260811 15:13:00.759907 1552 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:13:00.767973 1543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:00.758649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38561 (local address 127.30.8.62:44809)
0811 15:13:00.758770 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:00.758775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:00.759130 (+ 355us) server_negotiation.cc:407] Connection header received
0811 15:13:00.759276 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:00.759279 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:00.759331 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:00.759434 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:00.760406 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:00.761198 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:00.762350 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:00.762693 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:00.762776 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:00.764558 (+ 1782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:00.764562 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:00.764565 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:00.764600 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:00.766167 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:00.766697 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:00.766700 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:00.766702 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:00.766778 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:00.767099 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:00.767102 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:00.767103 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:00.767307 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:00.767431 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:00.767703 (+ 272us) server_negotiation.cc:299] Negotiation successful
0811 15:13:00.767801 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 15:13:00.771836 1552 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:13:00.772792 1552 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:13:00.783259 1552 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7b72c81bc502411ba82f420acdbed297: Generated new TSK 0
I20260811 15:13:00.784191 1552 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:13:00.796485 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 1420
W20260811 15:13:00.812098 1393 connection.cc:535] client connection to 127.30.8.62:44809 recv error: Network error: failed to read from TLS socket (remote: 127.30.8.62:44809): Connection reset by peer (error 104)
2026-08-11T15:13:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1610 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:13:00 dist-test-slave-1vmg krb5kdc[1585](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:13:00 dist-test-slave-1vmg krb5kdc[1585](info): set up 2 sockets
Aug 11 15:13:00 dist-test-slave-1vmg krb5kdc[1585](info): commencing operation
krb5kdc: starting...
W20260811 15:13:02.011030 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:13:02 dist-test-slave-1vmg krb5kdc[1585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:13:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:13:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:13:02.158123 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:42401
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:34729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:42401
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:13:02.298770 1607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:13:02.299189 1607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:13:02.299260 1607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:13:02.303777 1607 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:13:02.303893 1607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:13:02.303921 1607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:13:02.303946 1607 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:13:02.303966 1607 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:13:02.310281 1607 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:34729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:42401
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:42401
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:13:02.311870 1607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:13:02.313236 1607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:13:02.323890 1616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:02.323973 1617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:02.323894 1620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:13:02.324206 1607 server_base.cc:1034] running on GCE node
I20260811 15:13:02.325104 1607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:13:02.326673 1607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:13:02.327877 1607 hybrid_clock.cc:648] HybridClock initialized: now 1786461182327843 us; error 54 us; skew 500 ppm
Aug 11 15:13:02 dist-test-slave-1vmg krb5kdc[1585](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461182, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:13:02.330992 1607 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:13:02.332113 1607 webserver.cc:460] Webserver started at http://127.30.8.62:38479/ using document root <none> and password file <none>
I20260811 15:13:02.332784 1607 fs_manager.cc:359] Metadata directory not provided
I20260811 15:13:02.332859 1607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:13:02.333093 1607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:13:02.335505 1607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "c197ab24e7c142d79cbd6c6f9feec4a4"
format_stamp: "Formatted at 2026-08-11 15:13:02 on dist-test-slave-1vmg"
server_key: "c666c8a49aefad4680520815815b1ad7"
server_key_iv: "82dfe978155d0aac8016f4654aa26f07"
server_key_version: "encryptionkey@0"
I20260811 15:13:02.336114 1607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "c197ab24e7c142d79cbd6c6f9feec4a4"
format_stamp: "Formatted at 2026-08-11 15:13:02 on dist-test-slave-1vmg"
server_key: "c666c8a49aefad4680520815815b1ad7"
server_key_iv: "82dfe978155d0aac8016f4654aa26f07"
server_key_version: "encryptionkey@0"
I20260811 15:13:02.340318 1607 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:13:02.343050 1632 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:13:02.344472 1607 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:13:02.344627 1607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "c197ab24e7c142d79cbd6c6f9feec4a4"
format_stamp: "Formatted at 2026-08-11 15:13:02 on dist-test-slave-1vmg"
server_key: "c666c8a49aefad4680520815815b1ad7"
server_key_iv: "82dfe978155d0aac8016f4654aa26f07"
server_key_version: "encryptionkey@0"
I20260811 15:13:02.344746 1607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:13:02.377679 1607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:13:02.381256 1607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:13:02.381469 1607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:13:02.389249 1607 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:42401
I20260811 15:13:02.389264 1723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:42401 every 8 connection(s)
I20260811 15:13:02.390334 1607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:13:02.393455 1724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:13:02.398504 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 1607
I20260811 15:13:02.398674 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:13:02.399036 30752 external_mini_cluster.cc:1428] Setting key ec4ce28eb0c5876caa78223fab7130fd
I20260811 15:13:02.400171 1724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4: Bootstrap starting.
I20260811 15:13:02.404145 1724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4: Neither blocks nor log segments found. Creating new log.
I20260811 15:13:02.405185 1724 log.cc:824] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4: Log is configured to *not* fsync() on all Append() calls
I20260811 15:13:02.408056 1724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4: No bootstrap required, opened a new log
I20260811 15:13:02.409708 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:02.400567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52905 (local address 127.30.8.62:42401)
0811 15:13:02.401085 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:02.401102 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:02.401142 (+ 40us) server_negotiation.cc:407] Connection header received
0811 15:13:02.402554 (+ 1412us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:02.402582 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:02.403137 (+ 555us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:02.403517 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:02.403992 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:02.405054 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:02.405975 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:02.406498 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:02.406662 (+ 164us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:02.407746 (+ 1084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:02.407752 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:02.407770 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:13:02.408282 (+ 512us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:02.408346 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:13:02.408372 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:02.408495 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:02.408697 (+ 202us) server_negotiation.cc:299] Negotiation successful
0811 15:13:02.408861 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":172,"threads_started":1}
I20260811 15:13:02.413084 1724 raft_consensus.cc:348] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c197ab24e7c142d79cbd6c6f9feec4a4" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42401 } }
I20260811 15:13:02.413401 1724 raft_consensus.cc:374] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:13:02.413470 1724 raft_consensus.cc:729] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c197ab24e7c142d79cbd6c6f9feec4a4, State: Initialized, Role: FOLLOWER
I20260811 15:13:02.413700 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:13:02.414006 1724 consensus_queue.cc:260] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c197ab24e7c142d79cbd6c6f9feec4a4" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42401 } }
I20260811 15:13:02.414170 1724 raft_consensus.cc:388] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:13:02.414237 1724 raft_consensus.cc:482] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:13:02.414352 1724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:13:02.415673 1724 raft_consensus.cc:504] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c197ab24e7c142d79cbd6c6f9feec4a4" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42401 } }
I20260811 15:13:02.416132 1724 leader_election.cc:302] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c197ab24e7c142d79cbd6c6f9feec4a4; no voters:
I20260811 15:13:02.416636 1724 leader_election.cc:288] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:13:02.416756 1741 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:13:02.417079 1741 raft_consensus.cc:686] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4 [term 1 LEADER]: Becoming Leader. State: Replica: c197ab24e7c142d79cbd6c6f9feec4a4, State: Running, Role: LEADER
I20260811 15:13:02.417594 1741 consensus_queue.cc:237] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c197ab24e7c142d79cbd6c6f9feec4a4" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42401 } }
I20260811 15:13:02.418314 1724 sys_catalog.cc:564] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:13:02.419261 1741 sys_catalog.cc:455] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c197ab24e7c142d79cbd6c6f9feec4a4. Latest consensus state: current_term: 1 leader_uuid: "c197ab24e7c142d79cbd6c6f9feec4a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c197ab24e7c142d79cbd6c6f9feec4a4" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42401 } } }
I20260811 15:13:02.419417 1741 sys_catalog.cc:458] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4 [sys.catalog]: This master's current role is: LEADER
I20260811 15:13:02.419786 1742 sys_catalog.cc:455] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c197ab24e7c142d79cbd6c6f9feec4a4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c197ab24e7c142d79cbd6c6f9feec4a4" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 42401 } } }
I20260811 15:13:02.419905 1742 sys_catalog.cc:458] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4 [sys.catalog]: This master's current role is: LEADER
I20260811 15:13:02.421496 1729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:02.416014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43653 (local address 127.30.8.62:42401)
0811 15:13:02.416156 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:02.416160 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:02.416524 (+ 364us) server_negotiation.cc:407] Connection header received
0811 15:13:02.416578 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:02.416582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:02.416641 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:02.416725 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:02.417235 (+ 510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:02.417970 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:02.419185 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:02.419496 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:02.419581 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:02.420832 (+ 1251us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:02.420836 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:02.420838 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:13:02.420852 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:02.420880 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:13:02.420892 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:02.420971 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:02.421304 (+ 333us) server_negotiation.cc:299] Negotiation successful
0811 15:13:02.421346 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 15:13:02.424666 1765 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:13:02.424754 1765 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:13:02.424842 1750 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:13:02.425828 1750 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:13:02.432821 1750 catalog_manager.cc:1347] Generated new cluster ID: 98e9b35909af40448ed19793001286a4
I20260811 15:13:02.432924 1750 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:13:02.449149 1750 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:13:02.450593 1750 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:13:02.462889 1750 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c197ab24e7c142d79cbd6c6f9feec4a4: Generated new TSK 0
I20260811 15:13:02.463762 1750 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:13:02.478477 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 1607
W20260811 15:13:02.497750 1578 connection.cc:535] client connection to 127.30.8.62:42401 recv error: Network error: failed to read from TLS socket (remote: 127.30.8.62:42401): Connection reset by peer (error 104)
2026-08-11T15:13:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1685 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:13:02 dist-test-slave-1vmg krb5kdc[1775](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:13:02 dist-test-slave-1vmg krb5kdc[1775](info): set up 2 sockets
Aug 11 15:13:02 dist-test-slave-1vmg krb5kdc[1775](info): commencing operation
krb5kdc: starting...
W20260811 15:13:03.669253 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.157s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:13:03 dist-test-slave-1vmg krb5kdc[1775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:13:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:13:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:13:03.815197 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:40559
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:39955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:40559
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:13:03.947598 1799 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:13:03.947991 1799 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:13:03.948040 1799 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:13:03.952462 1799 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:13:03.952560 1799 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:13:03.952580 1799 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:13:03.952598 1799 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:13:03.952632 1799 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:13:03.958884 1799 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:39955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:40559
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:40559
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1799
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:13:03.960471 1799 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:13:03.961831 1799 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:13:03.972496 1805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:03.972506 1808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:03.972496 1812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:13:03.972995 1799 server_base.cc:1034] running on GCE node
I20260811 15:13:03.973788 1799 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:13:03.975226 1799 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:13:03.976384 1799 hybrid_clock.cc:648] HybridClock initialized: now 1786461183976366 us; error 45 us; skew 500 ppm
Aug 11 15:13:03 dist-test-slave-1vmg krb5kdc[1775](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461183, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:13:03.979313 1799 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:13:03.980389 1799 webserver.cc:460] Webserver started at http://127.30.8.62:42159/ using document root <none> and password file <none>
I20260811 15:13:03.980999 1799 fs_manager.cc:359] Metadata directory not provided
I20260811 15:13:03.981074 1799 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:13:03.981317 1799 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:13:03.983579 1799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "4f8bcd8a90e145418a237de7522e3ba4"
format_stamp: "Formatted at 2026-08-11 15:13:03 on dist-test-slave-1vmg"
server_key: "b425f8941392881378fee248e6b5643c"
server_key_iv: "f3416b7ef511b313aeaeb02fad727c11"
server_key_version: "encryptionkey@0"
I20260811 15:13:03.984212 1799 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "4f8bcd8a90e145418a237de7522e3ba4"
format_stamp: "Formatted at 2026-08-11 15:13:03 on dist-test-slave-1vmg"
server_key: "b425f8941392881378fee248e6b5643c"
server_key_iv: "f3416b7ef511b313aeaeb02fad727c11"
server_key_version: "encryptionkey@0"
I20260811 15:13:03.988425 1799 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:13:03.991161 1823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:13:03.992479 1799 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:13:03.992614 1799 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "4f8bcd8a90e145418a237de7522e3ba4"
format_stamp: "Formatted at 2026-08-11 15:13:03 on dist-test-slave-1vmg"
server_key: "b425f8941392881378fee248e6b5643c"
server_key_iv: "f3416b7ef511b313aeaeb02fad727c11"
server_key_version: "encryptionkey@0"
I20260811 15:13:03.992729 1799 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:13:03.998217 1799 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:13:04.001281 1799 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:13:04.001472 1799 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:13:04.009632 1799 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:40559
I20260811 15:13:04.009639 1904 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:40559 every 8 connection(s)
I20260811 15:13:04.010694 1799 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:13:04.013878 1906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:13:04.014463 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 1799
I20260811 15:13:04.014621 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:13:04.014883 30752 external_mini_cluster.cc:1428] Setting key 9e0fd2be39b8a23952d4c862cc9f4e16
I20260811 15:13:04.020236 1912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:04.016312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35267 (local address 127.30.8.62:40559)
0811 15:13:04.016791 (+ 479us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:04.016806 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:04.016844 (+ 38us) server_negotiation.cc:407] Connection header received
0811 15:13:04.017535 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:04.017554 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:04.017855 (+ 301us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:04.018199 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:04.019007 (+ 808us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:04.019012 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:04.019025 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:13:04.019035 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:04.019101 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:13:04.019121 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:04.019258 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:04.019329 (+ 71us) server_negotiation.cc:299] Negotiation successful
0811 15:13:04.019487 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":136,"threads_started":1}
I20260811 15:13:04.021232 1906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4: Bootstrap starting.
I20260811 15:13:04.023229 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:13:04.023788 1906 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4: Neither blocks nor log segments found. Creating new log.
I20260811 15:13:04.024565 1906 log.cc:824] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4: Log is configured to *not* fsync() on all Append() calls
I20260811 15:13:04.026825 1906 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4: No bootstrap required, opened a new log
I20260811 15:13:04.027213 1912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:04.025336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39597 (local address 127.30.8.62:40559)
0811 15:13:04.025443 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:04.025448 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:04.025632 (+ 184us) server_negotiation.cc:407] Connection header received
0811 15:13:04.025873 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:04.025876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:04.025912 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:04.025989 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:04.026686 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:04.026689 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:04.026692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:13:04.026704 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:04.026732 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:13:04.026744 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:04.026831 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:04.027041 (+ 210us) server_negotiation.cc:299] Negotiation successful
0811 15:13:04.027093 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260811 15:13:04.030884 1906 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8bcd8a90e145418a237de7522e3ba4" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 40559 } }
I20260811 15:13:04.031191 1906 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:13:04.031261 1906 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f8bcd8a90e145418a237de7522e3ba4, State: Initialized, Role: FOLLOWER
I20260811 15:13:04.031718 1906 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8bcd8a90e145418a237de7522e3ba4" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 40559 } }
I20260811 15:13:04.031847 1906 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:13:04.031909 1906 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:13:04.032008 1906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:13:04.033069 1906 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8bcd8a90e145418a237de7522e3ba4" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 40559 } }
I20260811 15:13:04.033416 1906 leader_election.cc:302] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f8bcd8a90e145418a237de7522e3ba4; no voters:
I20260811 15:13:04.033738 1906 leader_election.cc:288] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:13:04.033879 1925 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:13:04.034104 1925 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4 [term 1 LEADER]: Becoming Leader. State: Replica: 4f8bcd8a90e145418a237de7522e3ba4, State: Running, Role: LEADER
I20260811 15:13:04.034448 1925 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8bcd8a90e145418a237de7522e3ba4" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 40559 } }
I20260811 15:13:04.035079 1906 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:13:04.036271 1928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f8bcd8a90e145418a237de7522e3ba4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8bcd8a90e145418a237de7522e3ba4" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 40559 } } }
I20260811 15:13:04.036386 1928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4 [sys.catalog]: This master's current role is: LEADER
I20260811 15:13:04.036689 1929 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f8bcd8a90e145418a237de7522e3ba4. Latest consensus state: current_term: 1 leader_uuid: "4f8bcd8a90e145418a237de7522e3ba4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f8bcd8a90e145418a237de7522e3ba4" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 40559 } } }
I20260811 15:13:04.036782 1929 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4 [sys.catalog]: This master's current role is: LEADER
I20260811 15:13:04.036888 1946 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:13:04.040055 1946 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:13:04.045806 1946 catalog_manager.cc:1347] Generated new cluster ID: b24dd10017e24e5588b6bc0517f04799
I20260811 15:13:04.045902 1946 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:13:04.094400 1946 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:13:04.094806 1946 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:13:04.112062 1946 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f8bcd8a90e145418a237de7522e3ba4: Generated new TSK 0
I20260811 15:13:04.112934 1946 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:13:04.124083 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 1799
2026-08-11T15:13:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1641 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:13:04 dist-test-slave-1vmg krb5kdc[1966](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:13:04 dist-test-slave-1vmg krb5kdc[1966](info): set up 2 sockets
Aug 11 15:13:04 dist-test-slave-1vmg krb5kdc[1966](info): commencing operation
krb5kdc: starting...
W20260811 15:13:05.342008 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:13:05 dist-test-slave-1vmg krb5kdc[1966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:13:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:13:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:13:05.484067 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:46501
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:41511
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:46501
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:13:05.615477 1991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:13:05.615806 1991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:13:05.615869 1991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:13:05.620141 1991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:13:05.620239 1991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:13:05.620268 1991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:13:05.620290 1991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:13:05.620317 1991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:13:05.626101 1991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:41511
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:46501
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:46501
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1991
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:13:05.627475 1991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:13:05.628630 1991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 15:13:05.638355 1991 server_base.cc:1034] running on GCE node
W20260811 15:13:05.638317 1999 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:05.638484 1998 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:05.638602 2002 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:13:05.639389 1991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:13:05.640631 1991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:13:05.641803 1991 hybrid_clock.cc:648] HybridClock initialized: now 1786461185641784 us; error 44 us; skew 500 ppm
Aug 11 15:13:05 dist-test-slave-1vmg krb5kdc[1966](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461185, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:13:05.644773 1991 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:13:05.645830 1991 webserver.cc:460] Webserver started at http://127.30.8.62:37391/ using document root <none> and password file <none>
I20260811 15:13:05.646449 1991 fs_manager.cc:359] Metadata directory not provided
I20260811 15:13:05.646517 1991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:13:05.646746 1991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:13:05.649020 1991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "bb555eb10ce44f74ace092b1835936c7"
format_stamp: "Formatted at 2026-08-11 15:13:05 on dist-test-slave-1vmg"
server_key: "61a10ee7d1afe21e7b2b5163ab8ea4a8"
server_key_iv: "72c62ae75dfbbf7c3cbd3077195ece48"
server_key_version: "encryptionkey@0"
I20260811 15:13:05.649613 1991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "bb555eb10ce44f74ace092b1835936c7"
format_stamp: "Formatted at 2026-08-11 15:13:05 on dist-test-slave-1vmg"
server_key: "61a10ee7d1afe21e7b2b5163ab8ea4a8"
server_key_iv: "72c62ae75dfbbf7c3cbd3077195ece48"
server_key_version: "encryptionkey@0"
I20260811 15:13:05.653816 1991 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 15:13:05.656464 2011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:13:05.657789 1991 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260811 15:13:05.657928 1991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "bb555eb10ce44f74ace092b1835936c7"
format_stamp: "Formatted at 2026-08-11 15:13:05 on dist-test-slave-1vmg"
server_key: "61a10ee7d1afe21e7b2b5163ab8ea4a8"
server_key_iv: "72c62ae75dfbbf7c3cbd3077195ece48"
server_key_version: "encryptionkey@0"
I20260811 15:13:05.658043 1991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:13:05.679445 1991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:13:05.684074 1991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:13:05.684301 1991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:13:05.691879 1991 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:46501
I20260811 15:13:05.691893 2086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:46501 every 8 connection(s)
I20260811 15:13:05.692955 1991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:13:05.693953 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 1991
I20260811 15:13:05.694098 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:13:05.694372 30752 external_mini_cluster.cc:1428] Setting key 4b8b24cdfb85c83451017b4981a48e82
I20260811 15:13:05.697058 2090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:13:05.704576 2090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7: Bootstrap starting.
Aug 11 15:13:05 dist-test-slave-1vmg krb5kdc[1966](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461185, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:13:05.707266 2090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7: Neither blocks nor log segments found. Creating new log.
I20260811 15:13:05.708039 2090 log.cc:824] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7: Log is configured to *not* fsync() on all Append() calls
I20260811 15:13:05.710156 2090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7: No bootstrap required, opened a new log
I20260811 15:13:05.711018 2094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:05.696638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45571 (local address 127.30.8.62:46501)
0811 15:13:05.697238 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:05.697263 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:05.697325 (+ 62us) server_negotiation.cc:407] Connection header received
0811 15:13:05.698162 (+ 837us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:05.698186 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:05.698653 (+ 467us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:05.699115 (+ 462us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:05.700147 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:05.701252 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:05.702443 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:05.703014 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:05.703152 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:05.706034 (+ 2882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:05.706039 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:05.706056 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:05.706091 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:05.707929 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:05.708436 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:05.708442 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:05.708448 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:05.708568 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:05.708853 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:05.708856 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:05.708857 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:05.709310 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:05.709510 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:05.709838 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 15:13:05.710158 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":187,"threads_started":1}
I20260811 15:13:05.714105 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:13:05.714334 2090 raft_consensus.cc:348] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb555eb10ce44f74ace092b1835936c7" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46501 } }
I20260811 15:13:05.714576 2090 raft_consensus.cc:374] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:13:05.714643 2090 raft_consensus.cc:729] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bb555eb10ce44f74ace092b1835936c7, State: Initialized, Role: FOLLOWER
I20260811 15:13:05.715117 2090 consensus_queue.cc:260] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb555eb10ce44f74ace092b1835936c7" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46501 } }
I20260811 15:13:05.715263 2090 raft_consensus.cc:388] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:13:05.715333 2090 raft_consensus.cc:482] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:13:05.715440 2090 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:13:05.716646 2090 raft_consensus.cc:504] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb555eb10ce44f74ace092b1835936c7" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46501 } }
I20260811 15:13:05.717020 2090 leader_election.cc:302] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bb555eb10ce44f74ace092b1835936c7; no voters:
I20260811 15:13:05.717347 2090 leader_election.cc:288] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:13:05.717432 2108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:13:05.717662 2108 raft_consensus.cc:686] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7 [term 1 LEADER]: Becoming Leader. State: Replica: bb555eb10ce44f74ace092b1835936c7, State: Running, Role: LEADER
I20260811 15:13:05.718027 2108 consensus_queue.cc:237] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb555eb10ce44f74ace092b1835936c7" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46501 } }
I20260811 15:13:05.718782 2090 sys_catalog.cc:564] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:13:05.720260 2111 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bb555eb10ce44f74ace092b1835936c7. Latest consensus state: current_term: 1 leader_uuid: "bb555eb10ce44f74ace092b1835936c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb555eb10ce44f74ace092b1835936c7" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46501 } } }
I20260811 15:13:05.720382 2111 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7 [sys.catalog]: This master's current role is: LEADER
I20260811 15:13:05.719746 2110 sys_catalog.cc:455] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bb555eb10ce44f74ace092b1835936c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bb555eb10ce44f74ace092b1835936c7" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46501 } } }
I20260811 15:13:05.720737 2110 sys_catalog.cc:458] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7 [sys.catalog]: This master's current role is: LEADER
I20260811 15:13:05.724433 2094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:05.716286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46261 (local address 127.30.8.62:46501)
0811 15:13:05.716412 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:05.716416 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:05.716540 (+ 124us) server_negotiation.cc:407] Connection header received
0811 15:13:05.716672 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:05.716674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:05.716715 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:05.716797 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:05.717514 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:05.718187 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:05.719040 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:05.719341 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:05.719863 (+ 522us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:05.721143 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:05.721147 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:05.721150 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:05.721178 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:05.722614 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:05.723120 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:05.723124 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:05.723125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:05.723235 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:05.723510 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:05.723513 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:05.723515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:05.723747 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:05.723879 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:05.724132 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 15:13:05.724274 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 15:13:05.724552 2139 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:13:05.724660 2139 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:13:05.724737 2131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:13:05.725574 2131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:13:05.731926 2131 catalog_manager.cc:1347] Generated new cluster ID: f3fdfeb73de34360ad81484738795007
I20260811 15:13:05.732007 2131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:13:05.750291 2131 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:13:05.751775 2131 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:13:05.759996 2131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bb555eb10ce44f74ace092b1835936c7: Generated new TSK 0
I20260811 15:13:05.760977 2131 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:13:05.769610 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 1991
2026-08-11T15:13:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1647 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:13:05 dist-test-slave-1vmg krb5kdc[2145](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:13:05 dist-test-slave-1vmg krb5kdc[2145](info): set up 2 sockets
Aug 11 15:13:05 dist-test-slave-1vmg krb5kdc[2145](info): commencing operation
krb5kdc: starting...
W20260811 15:13:06.997516 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.196s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:13:07 dist-test-slave-1vmg krb5kdc[2145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461187, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:13:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:13:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:13:07.152087 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:46277
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:34109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:46277
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:13:07.276922 2165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:13:07.277267 2165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:13:07.277331 2165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:13:07.281634 2165 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:13:07.281734 2165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:13:07.281764 2165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:13:07.281785 2165 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:13:07.281806 2165 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:13:07.287688 2165 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:34109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:46277
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:46277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:13:07.288954 2165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:13:07.290035 2165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:13:07.299335 2174 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:07.299453 2176 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:07.299329 2180 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:13:07.299649 2165 server_base.cc:1034] running on GCE node
I20260811 15:13:07.300469 2165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:13:07.301775 2165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:13:07.302968 2165 hybrid_clock.cc:648] HybridClock initialized: now 1786461187302947 us; error 37 us; skew 500 ppm
Aug 11 15:13:07 dist-test-slave-1vmg krb5kdc[2145](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461187, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:13:07.312743 2165 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:13:07.314267 2165 webserver.cc:460] Webserver started at http://127.30.8.62:45755/ using document root <none> and password file <none>
I20260811 15:13:07.315011 2165 fs_manager.cc:359] Metadata directory not provided
I20260811 15:13:07.315101 2165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:13:07.315351 2165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:13:07.317672 2165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "a4052f72124f4739bc1e67895b54ee39"
format_stamp: "Formatted at 2026-08-11 15:13:07 on dist-test-slave-1vmg"
server_key: "6a935b0d947b6797165dd170d681b3b4"
server_key_iv: "0b811b69ca34329445a2fea4f238bd4a"
server_key_version: "encryptionkey@0"
I20260811 15:13:07.318280 2165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "a4052f72124f4739bc1e67895b54ee39"
format_stamp: "Formatted at 2026-08-11 15:13:07 on dist-test-slave-1vmg"
server_key: "6a935b0d947b6797165dd170d681b3b4"
server_key_iv: "0b811b69ca34329445a2fea4f238bd4a"
server_key_version: "encryptionkey@0"
I20260811 15:13:07.322510 2165 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260811 15:13:07.325232 2190 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:13:07.326673 2165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 15:13:07.326833 2165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "a4052f72124f4739bc1e67895b54ee39"
format_stamp: "Formatted at 2026-08-11 15:13:07 on dist-test-slave-1vmg"
server_key: "6a935b0d947b6797165dd170d681b3b4"
server_key_iv: "0b811b69ca34329445a2fea4f238bd4a"
server_key_version: "encryptionkey@0"
I20260811 15:13:07.327003 2165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:13:07.355135 2165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:13:07.358175 2165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:13:07.358383 2165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:13:07.365870 2165 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:46277
I20260811 15:13:07.365898 2282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:46277 every 8 connection(s)
I20260811 15:13:07.366967 2165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:13:07.369997 2283 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:13:07.375900 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 2165
I20260811 15:13:07.376045 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:13:07.376171 2283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39: Bootstrap starting.
I20260811 15:13:07.376332 30752 external_mini_cluster.cc:1428] Setting key 40b97127be514dbd3c77fb5afcab999e
I20260811 15:13:07.379120 2283 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39: Neither blocks nor log segments found. Creating new log.
I20260811 15:13:07.380125 2283 log.cc:824] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39: Log is configured to *not* fsync() on all Append() calls
I20260811 15:13:07.382474 2283 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39: No bootstrap required, opened a new log
I20260811 15:13:07.386946 2291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:07.377832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60199 (local address 127.30.8.62:46277)
0811 15:13:07.378310 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:07.378337 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:07.378383 (+ 46us) server_negotiation.cc:407] Connection header received
0811 15:13:07.379319 (+ 936us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:07.379348 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:07.379849 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:07.380260 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:07.380699 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:07.382073 (+ 1374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:07.383067 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:07.383770 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:07.383945 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:07.384704 (+ 759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:07.384711 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:07.384730 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:13:07.385281 (+ 551us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:07.385357 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:13:07.385391 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:07.385496 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:07.385600 (+ 104us) server_negotiation.cc:299] Negotiation successful
0811 15:13:07.385858 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":126,"threads_started":1}
I20260811 15:13:07.387291 2283 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4052f72124f4739bc1e67895b54ee39" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46277 } }
I20260811 15:13:07.387537 2283 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:13:07.387606 2283 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4052f72124f4739bc1e67895b54ee39, State: Initialized, Role: FOLLOWER
I20260811 15:13:07.388056 2283 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4052f72124f4739bc1e67895b54ee39" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46277 } }
I20260811 15:13:07.388195 2283 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:13:07.388262 2283 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:13:07.388366 2283 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:13:07.389470 2283 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4052f72124f4739bc1e67895b54ee39" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46277 } }
I20260811 15:13:07.389822 2283 leader_election.cc:302] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4052f72124f4739bc1e67895b54ee39; no voters:
I20260811 15:13:07.389993 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:13:07.390206 2283 leader_election.cc:288] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:13:07.390297 2297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:13:07.390558 2297 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39 [term 1 LEADER]: Becoming Leader. State: Replica: a4052f72124f4739bc1e67895b54ee39, State: Running, Role: LEADER
I20260811 15:13:07.390887 2297 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4052f72124f4739bc1e67895b54ee39" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46277 } }
I20260811 15:13:07.391661 2283 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:13:07.392524 2298 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4052f72124f4739bc1e67895b54ee39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4052f72124f4739bc1e67895b54ee39" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46277 } } }
I20260811 15:13:07.392668 2298 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39 [sys.catalog]: This master's current role is: LEADER
I20260811 15:13:07.393057 2319 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:13:07.393045 2300 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4052f72124f4739bc1e67895b54ee39. Latest consensus state: current_term: 1 leader_uuid: "a4052f72124f4739bc1e67895b54ee39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4052f72124f4739bc1e67895b54ee39" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46277 } } }
I20260811 15:13:07.393141 2300 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39 [sys.catalog]: This master's current role is: LEADER
I20260811 15:13:07.396497 2319 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:13:07.398268 2291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:07.392862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56339 (local address 127.30.8.62:46277)
0811 15:13:07.392980 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:07.392984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:07.393882 (+ 898us) server_negotiation.cc:407] Connection header received
0811 15:13:07.394090 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:07.394092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:07.394136 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:07.394200 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:07.394519 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:07.395315 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:07.396461 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:07.396781 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:07.396886 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:07.397789 (+ 903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:07.397793 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:07.397796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:13:07.397811 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:07.397839 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:13:07.397853 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:07.397923 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:07.398059 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 15:13:07.398124 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 15:13:07.402945 2319 catalog_manager.cc:1347] Generated new cluster ID: c1aa315dd7a24bedba09e11ddf00cab0
I20260811 15:13:07.403038 2319 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:13:07.438395 2319 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:13:07.439564 2319 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:13:07.446630 2319 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4052f72124f4739bc1e67895b54ee39: Generated new TSK 0
I20260811 15:13:07.447687 2319 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:13:07.452234 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 2165
2026-08-11T15:13:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1687 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:13:07 dist-test-slave-1vmg krb5kdc[2339](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:13:07 dist-test-slave-1vmg krb5kdc[2339](info): set up 2 sockets
Aug 11 15:13:07 dist-test-slave-1vmg krb5kdc[2339](info): commencing operation
krb5kdc: starting...
W20260811 15:13:08.690578 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.201s user 0.000s sys 0.010s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:13:08 dist-test-slave-1vmg krb5kdc[2339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:13:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:13:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:13:08.842907 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:45065
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:42823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:45065
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:13:08.966039 2360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:13:08.966382 2360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:13:08.966427 2360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:13:08.970729 2360 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:13:08.970834 2360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:13:08.970853 2360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:13:08.970870 2360 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:13:08.970887 2360 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:13:08.976714 2360 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:42823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:45065
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:45065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:13:08.977972 2360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:13:08.979132 2360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:13:08.989006 2370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:08.989212 2367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:08.989006 2368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:13:08.989575 2360 server_base.cc:1034] running on GCE node
I20260811 15:13:08.990254 2360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:13:08.991572 2360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:13:08.992748 2360 hybrid_clock.cc:648] HybridClock initialized: now 1786461188992734 us; error 33 us; skew 500 ppm
Aug 11 15:13:08 dist-test-slave-1vmg krb5kdc[2339](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461188, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:13:08.995644 2360 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:13:08.996676 2360 webserver.cc:460] Webserver started at http://127.30.8.62:34559/ using document root <none> and password file <none>
I20260811 15:13:08.997308 2360 fs_manager.cc:359] Metadata directory not provided
I20260811 15:13:08.997386 2360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:13:08.997620 2360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:13:08.999907 2360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "ff8a88ea711f401c8302903dfd336c76"
format_stamp: "Formatted at 2026-08-11 15:13:08 on dist-test-slave-1vmg"
server_key: "0db90c83c6c43f1eca36e9af786568b1"
server_key_iv: "1b11bbabc9f916b6628d251cf96533ee"
server_key_version: "encryptionkey@0"
I20260811 15:13:09.000506 2360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "ff8a88ea711f401c8302903dfd336c76"
format_stamp: "Formatted at 2026-08-11 15:13:08 on dist-test-slave-1vmg"
server_key: "0db90c83c6c43f1eca36e9af786568b1"
server_key_iv: "1b11bbabc9f916b6628d251cf96533ee"
server_key_version: "encryptionkey@0"
I20260811 15:13:09.005128 2360 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 15:13:09.008004 2383 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:13:09.009320 2360 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:13:09.009459 2360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "ff8a88ea711f401c8302903dfd336c76"
format_stamp: "Formatted at 2026-08-11 15:13:08 on dist-test-slave-1vmg"
server_key: "0db90c83c6c43f1eca36e9af786568b1"
server_key_iv: "1b11bbabc9f916b6628d251cf96533ee"
server_key_version: "encryptionkey@0"
I20260811 15:13:09.009578 2360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:13:09.014854 2360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:13:09.018115 2360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:13:09.018306 2360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:13:09.025632 2360 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:45065
I20260811 15:13:09.025652 2478 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:45065 every 8 connection(s)
I20260811 15:13:09.026681 2360 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:13:09.029743 2479 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:13:09.032229 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 2360
I20260811 15:13:09.032394 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:13:09.032696 30752 external_mini_cluster.cc:1428] Setting key 279326a9ecee1534e01cc385524f429b
I20260811 15:13:09.036505 2479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76: Bootstrap starting.
I20260811 15:13:09.038218 2482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:09.034295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50629 (local address 127.30.8.62:45065)
0811 15:13:09.034785 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:09.034801 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:09.034841 (+ 40us) server_negotiation.cc:407] Connection header received
0811 15:13:09.035548 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:09.035566 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:09.035865 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:09.036204 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:09.037017 (+ 813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:09.037030 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:09.037043 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:13:09.037053 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:09.037110 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:13:09.037131 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:09.037263 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:09.037336 (+ 73us) server_negotiation.cc:299] Negotiation successful
0811 15:13:09.037494 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":150,"threads_started":1}
I20260811 15:13:09.039021 2479 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76: Neither blocks nor log segments found. Creating new log.
I20260811 15:13:09.039808 2479 log.cc:824] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76: Log is configured to *not* fsync() on all Append() calls
I20260811 15:13:09.041098 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:13:09.041954 2479 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76: No bootstrap required, opened a new log
I20260811 15:13:09.044723 2482 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:09.043206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56029 (local address 127.30.8.62:45065)
0811 15:13:09.043328 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:09.043332 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:09.043520 (+ 188us) server_negotiation.cc:407] Connection header received
0811 15:13:09.043620 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:09.043623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:09.043659 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:09.043745 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:09.044371 (+ 626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:09.044375 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:09.044378 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 15:13:09.044391 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:09.044419 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 15:13:09.044433 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:09.044521 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:09.044559 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 15:13:09.044597 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 15:13:09.046214 2479 raft_consensus.cc:348] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff8a88ea711f401c8302903dfd336c76" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45065 } }
I20260811 15:13:09.046458 2479 raft_consensus.cc:374] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 15:13:09.046492 2479 raft_consensus.cc:729] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff8a88ea711f401c8302903dfd336c76, State: Initialized, Role: FOLLOWER
I20260811 15:13:09.046908 2479 consensus_queue.cc:260] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff8a88ea711f401c8302903dfd336c76" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45065 } }
I20260811 15:13:09.047181 2479 raft_consensus.cc:388] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 15:13:09.047235 2479 raft_consensus.cc:482] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 15:13:09.047314 2479 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:13:09.048274 2479 raft_consensus.cc:504] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff8a88ea711f401c8302903dfd336c76" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45065 } }
I20260811 15:13:09.048607 2479 leader_election.cc:302] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ff8a88ea711f401c8302903dfd336c76; no voters:
I20260811 15:13:09.048902 2479 leader_election.cc:288] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 15:13:09.049052 2491 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:13:09.049289 2491 raft_consensus.cc:686] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76 [term 1 LEADER]: Becoming Leader. State: Replica: ff8a88ea711f401c8302903dfd336c76, State: Running, Role: LEADER
I20260811 15:13:09.049634 2491 consensus_queue.cc:237] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff8a88ea711f401c8302903dfd336c76" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45065 } }
I20260811 15:13:09.050273 2479 sys_catalog.cc:564] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:13:09.051340 2493 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ff8a88ea711f401c8302903dfd336c76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff8a88ea711f401c8302903dfd336c76" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45065 } } }
I20260811 15:13:09.051491 2493 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76 [sys.catalog]: This master's current role is: LEADER
I20260811 15:13:09.051781 2494 sys_catalog.cc:455] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ff8a88ea711f401c8302903dfd336c76. Latest consensus state: current_term: 1 leader_uuid: "ff8a88ea711f401c8302903dfd336c76" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ff8a88ea711f401c8302903dfd336c76" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 45065 } } }
I20260811 15:13:09.051882 2494 sys_catalog.cc:458] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76 [sys.catalog]: This master's current role is: LEADER
I20260811 15:13:09.051983 2508 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:13:09.055577 2508 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:13:09.063071 2508 catalog_manager.cc:1347] Generated new cluster ID: d007b52ecf5f4af59093876b48ea07bc
I20260811 15:13:09.063205 2508 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:13:09.075336 2508 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:13:09.075856 2508 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:13:09.091252 2508 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ff8a88ea711f401c8302903dfd336c76: Generated new TSK 0
I20260811 15:13:09.092595 2508 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 15:13:09.107527 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 2360
2026-08-11T15:13:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1655 ms)
[----------] 18 tests from AuthTokenIssuingTest (29508 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:13:09 dist-test-slave-1vmg krb5kdc[2528](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:13:09 dist-test-slave-1vmg krb5kdc[2528](info): set up 2 sockets
Aug 11 15:13:09 dist-test-slave-1vmg krb5kdc[2528](info): commencing operation
krb5kdc: starting...
W20260811 15:13:10.328450 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.184s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:13:10 dist-test-slave-1vmg krb5kdc[2528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:13:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:13:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:13:10.470408 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:46877
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:40649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:46877,127.30.8.61:36145,127.30.8.60:43643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:13:10.596510 2552 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 15:13:10.596901 2552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:13:10.597337 2552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:13:10.597591 2552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:13:10.602873 2552 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:13:10.603010 2552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:13:10.603030 2552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:13:10.603048 2552 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:13:10.603065 2552 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:13:10.610387 2552 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:40649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:46877,127.30.8.61:36145,127.30.8.60:43643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:46877
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:13:10.611727 2552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:13:10.612818 2552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:13:10.622303 2560 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:10.622386 2567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:10.622386 2562 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:13:10.622960 2552 server_base.cc:1034] running on GCE node
I20260811 15:13:10.623656 2552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:13:10.625069 2552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:13:10.626266 2552 hybrid_clock.cc:648] HybridClock initialized: now 1786461190626254 us; error 38 us; skew 500 ppm
Aug 11 15:13:10 dist-test-slave-1vmg krb5kdc[2528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461190, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:13:10.630277 2552 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:13:10.631405 2552 webserver.cc:460] Webserver started at http://127.30.8.62:44759/ using document root <none> and password file <none>
I20260811 15:13:10.632030 2552 fs_manager.cc:359] Metadata directory not provided
I20260811 15:13:10.632105 2552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:13:10.632339 2552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:13:10.634634 2552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "59147afdeb8941d19683423bd5cd8df3"
format_stamp: "Formatted at 2026-08-11 15:13:10 on dist-test-slave-1vmg"
server_key: "700fb04b5b03b69e5ca03d8063cad6d7"
server_key_iv: "35f47e2d5af2762da3373e03d8c3135e"
server_key_version: "encryptionkey@0"
I20260811 15:13:10.635278 2552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "59147afdeb8941d19683423bd5cd8df3"
format_stamp: "Formatted at 2026-08-11 15:13:10 on dist-test-slave-1vmg"
server_key: "700fb04b5b03b69e5ca03d8063cad6d7"
server_key_iv: "35f47e2d5af2762da3373e03d8c3135e"
server_key_version: "encryptionkey@0"
I20260811 15:13:10.639609 2552 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 15:13:10.642472 2579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:13:10.643934 2552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 15:13:10.644162 2552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "59147afdeb8941d19683423bd5cd8df3"
format_stamp: "Formatted at 2026-08-11 15:13:10 on dist-test-slave-1vmg"
server_key: "700fb04b5b03b69e5ca03d8063cad6d7"
server_key_iv: "35f47e2d5af2762da3373e03d8c3135e"
server_key_version: "encryptionkey@0"
I20260811 15:13:10.644299 2552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:13:10.666653 2552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:13:10.670152 2552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:13:10.670333 2552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:13:10.678316 2552 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:46877
I20260811 15:13:10.678613 2677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:46877 every 8 connection(s)
I20260811 15:13:10.679553 2552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:13:10.682739 2679 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:13:10.687090 2679 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } has no permanent_uuid. Determining permanent_uuid...
I20260811 15:13:10.689733 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 2552
I20260811 15:13:10.689895 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:13:10.690182 30752 external_mini_cluster.cc:1428] Setting key 5a259a6171299cb4768a17aa49e0fcfd
Aug 11 15:13:10 dist-test-slave-1vmg krb5kdc[2528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461190, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.8.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.61@KRBTEST.COM" created.
I20260811 15:13:10.704612 2685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:10.690522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54678 (local address 127.30.8.62:46877)
0811 15:13:10.690831 (+ 309us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:10.690842 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:10.690889 (+ 47us) server_negotiation.cc:407] Connection header received
0811 15:13:10.691527 (+ 638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:10.691533 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:10.691791 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:10.692001 (+ 210us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:10.693019 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:10.694041 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:10.695125 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:10.695770 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:10.695961 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:10.698345 (+ 2384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:10.698354 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:10.698371 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:10.698400 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:10.700269 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:10.700740 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:10.700746 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:10.700748 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:10.700874 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:10.701335 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:10.701341 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:10.701342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:10.702104 (+ 762us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:10.702382 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:10.702967 (+ 585us) server_negotiation.cc:299] Negotiation successful
0811 15:13:10.703561 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":249,"server-negotiator.queue_time_us":199,"thread_start_us":94,"threads_started":1}
I20260811 15:13:10.704618 2683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:10.689552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:46877 (local address 127.0.0.1:54678)
0811 15:13:10.690118 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:10.690175 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 15:13:10.691196 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:13:10.692003 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:13:10.692007 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:13:10.692465 (+ 458us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:13:10.692855 (+ 390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:10.692876 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:10.694173 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:10.694176 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:10.695010 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:10.695021 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:10.695930 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:10.695933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:10.696016 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:10.696635 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:13:10.696653 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:13:10.698172 (+ 1519us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:13:10.700382 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:10.700391 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:10.700402 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:10.700640 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:10.700964 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:10.700967 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:10.700969 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:10.701100 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:10.702321 (+ 1221us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:13:10.702330 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:13:10.702532 (+ 202us) client_negotiation.cc:769] Sending connection context
0811 15:13:10.702812 (+ 280us) client_negotiation.cc:240] Negotiation successful
0811 15:13:10.703445 (+ 633us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"mutex_wait_us":31,"spinlock_wait_cycles":13824,"thread_start_us":146,"threads_started":1}
I20260811 15:13:10.708220 2679 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } has no permanent_uuid. Determining permanent_uuid...
I20260811 15:13:10.709126 2683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:10.708811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:36145 (local address 127.0.0.1:36432)
0811 15:13:10.708961 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:10.709047 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.61:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 15:13:10.709373 2583 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.8.61:36145: connect: Connection refused (error 111)
I20260811 15:13:10.710490 2683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:10.710264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:36145 (local address 127.0.0.1:36448)
0811 15:13:10.710385 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:10.710432 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.61:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 15:13:10.710631 2679 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.61:36145: Network error: Client connection negotiation failed: client connection to 127.30.8.61:36145: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.61.keytab.
Entry for principal kudu/127.30.8.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.61.keytab.
I20260811 15:13:10.726692 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.61:36145
--webserver_interface=127.30.8.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.61.keytab
--principal=kudu/127.30.8.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:40649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:46877,127.30.8.61:36145,127.30.8.60:43643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:13:10.759904 2679 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } attempt: 1
I20260811 15:13:10.760908 2683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:10.760591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:36145 (local address 127.0.0.1:36452)
0811 15:13:10.760754 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:10.760827 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.61:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 15:13:10.761673 2683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:10.761459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:36145 (local address 127.0.0.1:36460)
0811 15:13:10.761577 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:10.761621 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.61:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 15:13:10.761806 2679 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.61:36145: Network error: Client connection negotiation failed: client connection to 127.30.8.61:36145: connect: Connection refused (error 111)
I20260811 15:13:10.830006 2679 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } attempt: 2
I20260811 15:13:10.831027 2683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:10.830684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:36145 (local address 127.0.0.1:36468)
0811 15:13:10.830844 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:10.830907 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.61:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 15:13:10.831741 2683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:10.831533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:36145 (local address 127.0.0.1:36474)
0811 15:13:10.831641 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:10.831689 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.61:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 15:13:10.831869 2679 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.61:36145: Network error: Client connection negotiation failed: client connection to 127.30.8.61:36145: connect: Connection refused (error 111)
W20260811 15:13:10.849777 2692 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 15:13:10.850150 2692 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:13:10.850196 2692 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:13:10.850263 2692 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:13:10.854537 2692 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:13:10.854640 2692 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:13:10.855263 2692 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:13:10.855386 2692 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:13:10.855417 2692 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:13:10.863032 2692 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:40649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:46877,127.30.8.61:36145,127.30.8.60:43643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.61.keytab
--principal=kudu/127.30.8.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.61:36145
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.8.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2692
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:13:10.864702 2692 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:13:10.866132 2692 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:13:10.876384 2709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:13:10.876621 2692 server_base.cc:1034] running on GCE node
W20260811 15:13:10.876382 2706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:10.876382 2705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:13:10.877450 2692 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:13:10.878760 2692 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:13:10.879946 2692 hybrid_clock.cc:648] HybridClock initialized: now 1786461190879919 us; error 49 us; skew 500 ppm
Aug 11 15:13:10 dist-test-slave-1vmg krb5kdc[2528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461190, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:13:10.883754 2692 init.cc:377] Logged in from keytab as kudu/127.30.8.61@KRBTEST.COM (short username kudu)
I20260811 15:13:10.885231 2692 webserver.cc:460] Webserver started at http://127.30.8.61:37711/ using document root <none> and password file <none>
I20260811 15:13:10.886010 2692 fs_manager.cc:359] Metadata directory not provided
I20260811 15:13:10.886093 2692 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:13:10.886370 2692 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:13:10.889305 2692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-1/data/instance:
uuid: "4b65033630dc4fd2bdc79068b3517cb3"
format_stamp: "Formatted at 2026-08-11 15:13:10 on dist-test-slave-1vmg"
server_key: "096c5470c6ed25c127337011050057c1"
server_key_iv: "e6f0c0011be33e3fb11b770cb3ea1c0d"
server_key_version: "encryptionkey@0"
I20260811 15:13:10.890010 2692 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-1/wal/instance:
uuid: "4b65033630dc4fd2bdc79068b3517cb3"
format_stamp: "Formatted at 2026-08-11 15:13:10 on dist-test-slave-1vmg"
server_key: "096c5470c6ed25c127337011050057c1"
server_key_iv: "e6f0c0011be33e3fb11b770cb3ea1c0d"
server_key_version: "encryptionkey@0"
I20260811 15:13:10.895430 2692 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260811 15:13:10.899166 2720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:13:10.901002 2692 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 15:13:10.901275 2692 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-1/wal
uuid: "4b65033630dc4fd2bdc79068b3517cb3"
format_stamp: "Formatted at 2026-08-11 15:13:10 on dist-test-slave-1vmg"
server_key: "096c5470c6ed25c127337011050057c1"
server_key_iv: "e6f0c0011be33e3fb11b770cb3ea1c0d"
server_key_version: "encryptionkey@0"
I20260811 15:13:10.901481 2692 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:13:10.920885 2692 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:13:10.923033 2679 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } attempt: 3
I20260811 15:13:10.924137 2683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:10.923749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:36145 (local address 127.0.0.1:36486)
0811 15:13:10.923957 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:10.924036 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.61:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 15:13:10.924721 2692 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:13:10.924892 2692 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:13:10.925225 2683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:10.924927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:36145 (local address 127.0.0.1:36490)
0811 15:13:10.925070 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:10.925131 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.61:36145: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 15:13:10.925387 2679 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.61:36145: Network error: Client connection negotiation failed: client connection to 127.30.8.61:36145: connect: Connection refused (error 111)
I20260811 15:13:10.932603 2692 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.61:36145
I20260811 15:13:10.932911 2814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.61:36145 every 8 connection(s)
I20260811 15:13:10.933669 2692 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-1/data/info.pb
I20260811 15:13:10.935786 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 2692
I20260811 15:13:10.935931 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-1/wal/instance
I20260811 15:13:10.936267 30752 external_mini_cluster.cc:1428] Setting key 23467e5aecc70feb0d195a3b2f2a7deb
I20260811 15:13:10.936945 2815 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:13:10.942836 2815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.8.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.60@KRBTEST.COM" created.
Aug 11 15:13:10 dist-test-slave-1vmg krb5kdc[2528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461190, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.61@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:13:10.959234 2685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:10.945643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54692 (local address 127.30.8.62:46877)
0811 15:13:10.947071 (+ 1428us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:10.947077 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:10.947095 (+ 18us) server_negotiation.cc:407] Connection header received
0811 15:13:10.947981 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:10.947985 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:10.948046 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:10.948124 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:10.949437 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:10.949981 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:10.951377 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:10.951641 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:10.951752 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:10.954501 (+ 2749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:10.954505 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:10.954507 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:10.954536 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:10.956108 (+ 1572us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:10.957100 (+ 992us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:10.957104 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:10.957107 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:10.957229 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:10.957646 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:10.957650 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:10.957651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:10.957936 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:10.958060 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:10.958827 (+ 767us) server_negotiation.cc:299] Negotiation successful
0811 15:13:10.959029 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1332}
I20260811 15:13:10.960422 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:10.946146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:46877 (local address 127.0.0.1:54692)
0811 15:13:10.946749 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:10.946795 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 15:13:10.947758 (+ 963us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:13:10.948303 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:13:10.948310 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:13:10.948719 (+ 409us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:13:10.949298 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:10.949313 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:10.950112 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:10.950115 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:10.951250 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:10.951262 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:10.951830 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:10.951833 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:10.952003 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:10.952608 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:13:10.952627 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:13:10.954273 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:13:10.956366 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:10.956376 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:10.956404 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:10.956907 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:10.957344 (+ 437us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:10.957347 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:10.957348 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:10.957536 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:10.958072 (+ 536us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:13:10.958079 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:13:10.958502 (+ 423us) client_negotiation.cc:769] Sending connection context
0811 15:13:10.958716 (+ 214us) client_negotiation.cc:240] Negotiation successful
0811 15:13:10.959190 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":187,"threads_started":1}
I20260811 15:13:10.962455 2815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 15:13:10 dist-test-slave-1vmg krb5kdc[2528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461190, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.61@KRBTEST.COM for kudu/127.30.8.61@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.60.keytab.
Entry for principal kudu/127.30.8.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.60.keytab.
I20260811 15:13:10.970840 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.60:43643
--webserver_interface=127.30.8.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.60.keytab
--principal=kudu/127.30.8.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:40649
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:46877,127.30.8.61:36145,127.30.8.60:43643
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:13:10.972623 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:10.963104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:36145 (local address 127.0.0.1:36494)
0811 15:13:10.963292 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:10.963307 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:13:10.963392 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:13:10.964322 (+ 930us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:13:10.964325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:13:10.964536 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:13:10.964671 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:10.964678 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:10.965730 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:10.965733 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:10.966321 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:10.966330 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:10.967074 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:10.967078 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:10.967169 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:10.967727 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:13:10.967739 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:13:10.969170 (+ 1431us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:13:10.971063 (+ 1893us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:10.971066 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:10.971068 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:10.971257 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:10.971537 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:10.971540 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:10.971542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:10.971615 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:10.972096 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:13:10.972099 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:13:10.972186 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 15:13:10.972274 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 15:13:10.972425 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54,"spinlock_wait_cycles":2816}
I20260811 15:13:10.972867 2825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:10.963307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36494 (local address 127.30.8.61:36145)
0811 15:13:10.963734 (+ 427us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:10.963747 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:10.963774 (+ 27us) server_negotiation.cc:407] Connection header received
0811 15:13:10.963818 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:10.963823 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:10.964165 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:10.964328 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:10.964778 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:10.965634 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:10.966431 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:10.966965 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:10.967064 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:10.969312 (+ 2248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:10.969323 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:10.969341 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:10.969362 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:10.970935 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:10.971364 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:10.971370 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:10.971372 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:10.971454 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:10.971694 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:10.971698 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:10.971700 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:10.971987 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:10.972119 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:10.972380 (+ 261us) server_negotiation.cc:299] Negotiation successful
0811 15:13:10.972521 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"spinlock_wait_cycles":2304,"thread_start_us":179,"threads_started":1}
I20260811 15:13:10.974750 2815 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } has no permanent_uuid. Determining permanent_uuid...
I20260811 15:13:10.975891 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:10.975483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:43643 (local address 127.0.0.1:39728)
0811 15:13:10.975658 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:10.975786 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:43643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
W20260811 15:13:10.976171 2722 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.8.60:43643: connect: Connection refused (error 111)
I20260811 15:13:10.977469 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:10.977186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:43643 (local address 127.0.0.1:39740)
0811 15:13:10.977338 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:10.977397 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:43643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
W20260811 15:13:10.977666 2815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.60:43643: Network error: Client connection negotiation failed: client connection to 127.30.8.60:43643: connect: Connection refused (error 111)
I20260811 15:13:11.027050 2815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } attempt: 1
I20260811 15:13:11.028019 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.027701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:43643 (local address 127.0.0.1:39750)
0811 15:13:11.027864 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:11.027929 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:43643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 15:13:11.028723 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.028532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:43643 (local address 127.0.0.1:39766)
0811 15:13:11.028625 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:11.028668 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:43643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 15:13:11.028842 2815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.60:43643: Network error: Client connection negotiation failed: client connection to 127.30.8.60:43643: connect: Connection refused (error 111)
I20260811 15:13:11.068620 2679 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } attempt: 4
Aug 11 15:13:11 dist-test-slave-1vmg krb5kdc[2528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461190, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for kudu/127.30.8.61@KRBTEST.COM
I20260811 15:13:11.077905 2825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.069506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36506 (local address 127.30.8.61:36145)
0811 15:13:11.069648 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.069652 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.069668 (+ 16us) server_negotiation.cc:407] Connection header received
0811 15:13:11.069829 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.069832 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.069881 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.069949 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.070413 (+ 464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.070983 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.071745 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.071989 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.072087 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:11.074160 (+ 2073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:11.074163 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:11.074165 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:11.074190 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:11.076275 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.076708 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.076711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.076713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.076790 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.077053 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.077056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.077058 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.077255 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:11.077343 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:11.077612 (+ 269us) server_negotiation.cc:299] Negotiation successful
0811 15:13:11.077726 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 15:13:11.077905 2683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.069346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:36145 (local address 127.0.0.1:36506)
0811 15:13:11.069591 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:11.069610 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 15:13:11.069723 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:13:11.069993 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:13:11.069995 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:13:11.070192 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:13:11.070289 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:11.070295 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.071100 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.071104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:11.071629 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:11.071638 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.072099 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.072103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:11.072223 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:11.072735 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:13:11.072747 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:13:11.074015 (+ 1268us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:13:11.076389 (+ 2374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:11.076393 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:11.076395 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:11.076612 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:11.076875 (+ 263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:11.076879 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:11.076881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:11.076960 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:11.077351 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:13:11.077354 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:13:11.077452 (+ 98us) client_negotiation.cc:769] Sending connection context
0811 15:13:11.077549 (+ 97us) client_negotiation.cc:240] Negotiation successful
0811 15:13:11.077717 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130}
I20260811 15:13:11.078774 2679 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } has no permanent_uuid. Determining permanent_uuid...
I20260811 15:13:11.079584 2683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.079320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:43643 (local address 127.0.0.1:39780)
0811 15:13:11.079452 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:11.079505 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:43643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 15:13:11.080256 2683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.080066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:43643 (local address 127.0.0.1:39792)
0811 15:13:11.080162 (+ 96us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:11.080207 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:43643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 15:13:11.080363 2679 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.60:43643: Network error: Client connection negotiation failed: client connection to 127.30.8.60:43643: connect: Connection refused (error 111)
W20260811 15:13:11.094856 2826 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 15:13:11.095283 2826 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:13:11.095328 2826 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:13:11.095394 2826 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 15:13:11.096992 2815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } attempt: 2
I20260811 15:13:11.097886 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.097587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:43643 (local address 127.0.0.1:39802)
0811 15:13:11.097745 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:11.097808 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:43643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 15:13:11.098546 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.098345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:43643 (local address 127.0.0.1:39814)
0811 15:13:11.098446 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:11.098489 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:43643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260811 15:13:11.098652 2815 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.60:43643: Network error: Client connection negotiation failed: client connection to 127.30.8.60:43643: connect: Connection refused (error 111)
W20260811 15:13:11.099978 2826 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:13:11.100041 2826 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:13:11.100069 2826 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:13:11.100101 2826 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:13:11.100131 2826 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:13:11.105990 2826 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:40649
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:46877,127.30.8.61:36145,127.30.8.60:43643
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.60.keytab
--principal=kudu/127.30.8.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.60:43643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.8.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2826
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:13:11.107406 2826 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:13:11.108604 2826 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:13:11.117826 2839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:11.118001 2836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:11.118001 2834 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:13:11.118301 2826 server_base.cc:1034] running on GCE node
I20260811 15:13:11.119022 2826 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:13:11.120333 2826 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:13:11.121515 2826 hybrid_clock.cc:648] HybridClock initialized: now 1786461191121492 us; error 54 us; skew 500 ppm
Aug 11 15:13:11 dist-test-slave-1vmg krb5kdc[2528](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461191, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:13:11.124938 2826 init.cc:377] Logged in from keytab as kudu/127.30.8.60@KRBTEST.COM (short username kudu)
I20260811 15:13:11.126015 2826 webserver.cc:460] Webserver started at http://127.30.8.60:42611/ using document root <none> and password file <none>
I20260811 15:13:11.126648 2826 fs_manager.cc:359] Metadata directory not provided
I20260811 15:13:11.126734 2826 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:13:11.127044 2826 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:13:11.129331 2826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-2/data/instance:
uuid: "7aba1fd582d640ab8e481cd4b83d9413"
format_stamp: "Formatted at 2026-08-11 15:13:11 on dist-test-slave-1vmg"
server_key: "81f6b40bf8c8a9a75daeb6ed3ca7eb3e"
server_key_iv: "abde3d861bd98c5feb71b48db202b85e"
server_key_version: "encryptionkey@0"
I20260811 15:13:11.129937 2826 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-2/wal/instance:
uuid: "7aba1fd582d640ab8e481cd4b83d9413"
format_stamp: "Formatted at 2026-08-11 15:13:11 on dist-test-slave-1vmg"
server_key: "81f6b40bf8c8a9a75daeb6ed3ca7eb3e"
server_key_iv: "abde3d861bd98c5feb71b48db202b85e"
server_key_version: "encryptionkey@0"
I20260811 15:13:11.134150 2826 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 15:13:11.137421 2853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:13:11.139415 2826 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 15:13:11.139611 2826 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-2/wal
uuid: "7aba1fd582d640ab8e481cd4b83d9413"
format_stamp: "Formatted at 2026-08-11 15:13:11 on dist-test-slave-1vmg"
server_key: "81f6b40bf8c8a9a75daeb6ed3ca7eb3e"
server_key_iv: "abde3d861bd98c5feb71b48db202b85e"
server_key_version: "encryptionkey@0"
I20260811 15:13:11.139606 2679 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } attempt: 1
I20260811 15:13:11.139793 2826 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:13:11.140666 2683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.140204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:43643 (local address 127.0.0.1:39816)
0811 15:13:11.140435 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:11.140542 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:43643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
I20260811 15:13:11.141551 2683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.141289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:43643 (local address 127.0.0.1:39822)
0811 15:13:11.141413 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:11.141475 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:43643: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 15:13:11.141716 2679 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.60:43643: Network error: Client connection negotiation failed: client connection to 127.30.8.60:43643: connect: Connection refused (error 111)
I20260811 15:13:11.156750 2826 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:13:11.159971 2826 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:13:11.160161 2826 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:13:11.167109 2826 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.60:43643
I20260811 15:13:11.167402 2952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.60:43643 every 8 connection(s)
I20260811 15:13:11.168262 2826 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-2/data/info.pb
I20260811 15:13:11.169538 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 2826
I20260811 15:13:11.169654 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786460893564915-30752-0/minicluster-data/master-2/wal/instance
I20260811 15:13:11.169935 30752 external_mini_cluster.cc:1428] Setting key abdc9e21d2e2838d77849cc7168dc114
I20260811 15:13:11.172739 2953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:13:11.177865 2953 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 15:13:11 dist-test-slave-1vmg krb5kdc[2528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.60@KRBTEST.COM
Aug 11 15:13:11 dist-test-slave-1vmg krb5kdc[2528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461191, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.60@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:13:11.187798 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.171664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39826 (local address 127.30.8.60:43643)
0811 15:13:11.172308 (+ 644us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.172328 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.172378 (+ 50us) server_negotiation.cc:407] Connection header received
0811 15:13:11.173177 (+ 799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.173201 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.173624 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.174003 (+ 379us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.174956 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.176080 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.176966 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.177514 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.177810 (+ 296us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:11.181460 (+ 3650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:11.181466 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:11.181481 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:11.181517 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:11.184283 (+ 2766us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.184916 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.184922 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.184929 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.185073 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.185363 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.185366 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.185367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.185854 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:11.186057 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:11.186391 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 15:13:11.186729 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":221,"threads_started":1}
I20260811 15:13:11.189802 2815 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } attempt: 3
I20260811 15:13:11.191819 2685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.180505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54706 (local address 127.30.8.62:46877)
0811 15:13:11.180678 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.180684 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.181166 (+ 482us) server_negotiation.cc:407] Connection header received
0811 15:13:11.181394 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.181397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.181453 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.181522 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.182621 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.183459 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.184745 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.184972 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.185067 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:11.187508 (+ 2441us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:11.187512 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:11.187514 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:11.187543 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:11.189681 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.190285 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.190289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.190291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.190388 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.190703 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.190707 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.190708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.190988 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:11.191080 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:11.191535 (+ 455us) server_negotiation.cc:299] Negotiation successful
0811 15:13:11.191659 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 15:13:11.191856 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.180395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:46877 (local address 127.0.0.1:54706)
0811 15:13:11.181010 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:11.181062 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 15:13:11.181233 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:13:11.181578 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:13:11.181584 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:13:11.182280 (+ 696us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:13:11.182464 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:11.182472 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.183664 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.183670 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:11.184611 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:11.184621 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.185089 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.185093 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:11.185216 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:11.185753 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:13:11.185765 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:13:11.187327 (+ 1562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:13:11.189825 (+ 2498us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:11.189833 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:11.189838 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:11.190154 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:11.190502 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:11.190505 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:11.190507 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:11.190599 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:11.191163 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:13:11.191169 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:13:11.191307 (+ 138us) client_negotiation.cc:769] Sending connection context
0811 15:13:11.191452 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 15:13:11.191660 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":154,"threads_started":1}
I20260811 15:13:11.193153 2953 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 15:13:11 dist-test-slave-1vmg krb5kdc[2528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461190, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.61@KRBTEST.COM for kudu/127.30.8.60@KRBTEST.COM
Aug 11 15:13:11 dist-test-slave-1vmg krb5kdc[2528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
Aug 11 15:13:11 dist-test-slave-1vmg krb5kdc[2528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461191, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.60@KRBTEST.COM for kudu/127.30.8.61@KRBTEST.COM
I20260811 15:13:11.198698 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.190258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:43643 (local address 127.0.0.1:39832)
0811 15:13:11.190447 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:11.190461 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 15:13:11.190537 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:13:11.190828 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:13:11.190832 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:13:11.191058 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:13:11.191155 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:11.191161 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.192430 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.192434 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:11.193026 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:11.193038 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.193517 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.193521 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:11.193616 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:11.194086 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:13:11.194101 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:13:11.195562 (+ 1461us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:13:11.197282 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:11.197285 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:11.197288 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:11.197481 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:11.197776 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:11.197779 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:11.197781 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:11.197866 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:11.198227 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:13:11.198230 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:13:11.198323 (+ 93us) client_negotiation.cc:769] Sending connection context
0811 15:13:11.198416 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 15:13:11.198560 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260811 15:13:11.199404 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.190347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39832 (local address 127.30.8.60:43643)
0811 15:13:11.190479 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.190483 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.190494 (+ 11us) server_negotiation.cc:407] Connection header received
0811 15:13:11.190646 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.190650 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.190708 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.191255 (+ 547us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.191346 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.192287 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.193146 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.193407 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.193493 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:11.195687 (+ 2194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:11.195692 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:11.195694 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:11.195719 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:11.197184 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.197578 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.197582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.197584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.197670 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.197950 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.197953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.197955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.198135 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:11.198221 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:11.199117 (+ 896us) server_negotiation.cc:299] Negotiation successful
0811 15:13:11.199257 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 15:13:11.203135 2979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.190687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54714 (local address 127.30.8.62:46877)
0811 15:13:11.191043 (+ 356us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.191057 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.191069 (+ 12us) server_negotiation.cc:407] Connection header received
0811 15:13:11.191103 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.191106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.191164 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.191249 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.192036 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.192781 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.194670 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.195079 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.195166 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:11.198666 (+ 3500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:11.198673 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:11.198676 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:11.198725 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:11.200789 (+ 2064us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.201611 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.201615 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.201617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.201725 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.202066 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.202070 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.202072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.202312 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:11.202533 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:11.202762 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 15:13:11.202908 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":95,"threads_started":1}
I20260811 15:13:11.203158 2825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.193735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36510 (local address 127.30.8.61:36145)
0811 15:13:11.193834 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.193838 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.193870 (+ 32us) server_negotiation.cc:407] Connection header received
0811 15:13:11.193992 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.193995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.194034 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.194095 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.194483 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.195167 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.196089 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.196305 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.196366 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:11.199081 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:11.199086 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:11.199089 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:11.199122 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:11.200979 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.201600 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.201605 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.201607 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.201700 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.202023 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.202027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.202029 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.202323 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:11.202396 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:11.202773 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 15:13:11.202965 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 15:13:11.203711 2815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3: Bootstrap starting.
I20260811 15:13:11.203949 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.193661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:36145 (local address 127.0.0.1:36510)
0811 15:13:11.193818 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:11.193830 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 15:13:11.193908 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:13:11.194119 (+ 211us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:13:11.194121 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:13:11.194294 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:13:11.194382 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:11.194388 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.195299 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.195302 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:11.195954 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:11.195965 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.196514 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.196518 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:11.196615 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:11.196970 (+ 355us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:13:11.196985 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:13:11.198826 (+ 1841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:13:11.201134 (+ 2308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:11.201138 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:11.201141 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:11.201470 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:11.201795 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:11.201798 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:11.201800 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:11.201911 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:11.202475 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:13:11.202478 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:13:11.202579 (+ 101us) client_negotiation.cc:769] Sending connection context
0811 15:13:11.203541 (+ 962us) client_negotiation.cc:240] Negotiation successful
0811 15:13:11.203719 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 15:13:11.205149 2953 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } has no permanent_uuid. Determining permanent_uuid...
I20260811 15:13:11.206857 2815 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3: Neither blocks nor log segments found. Creating new log.
I20260811 15:13:11.207715 2815 log.cc:824] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3: Log is configured to *not* fsync() on all Append() calls
I20260811 15:13:11.208863 2679 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } attempt: 2
I20260811 15:13:11.210462 2815 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3: No bootstrap required, opened a new log
Aug 11 15:13:11 dist-test-slave-1vmg krb5kdc[2528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461191, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.60@KRBTEST.COM for kudu/127.30.8.60@KRBTEST.COM
Aug 11 15:13:11 dist-test-slave-1vmg krb5kdc[2528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.61@KRBTEST.COM
Aug 11 15:13:11 dist-test-slave-1vmg krb5kdc[2528](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461190, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for kudu/127.30.8.60@KRBTEST.COM
I20260811 15:13:11.214977 2978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.205773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:43643 (local address 127.0.0.1:39834)
0811 15:13:11.205922 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:11.205937 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:13:11.206026 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:13:11.206478 (+ 452us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:13:11.206482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:13:11.206723 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:13:11.206893 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:11.206900 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.207851 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.207854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:11.208632 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:11.208642 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.209446 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.209450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:11.209524 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:11.209909 (+ 385us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:13:11.209923 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:13:11.211385 (+ 1462us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:13:11.213197 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:11.213201 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:11.213203 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:11.213421 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:11.213675 (+ 254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:11.213678 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:11.213680 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:11.213757 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:11.214423 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:13:11.214426 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:13:11.214518 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 15:13:11.214606 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 15:13:11.214748 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260811 15:13:11.215660 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.206055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39834 (local address 127.30.8.60:43643)
0811 15:13:11.206197 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.206202 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.206218 (+ 16us) server_negotiation.cc:407] Connection header received
0811 15:13:11.206261 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.206264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.206309 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.206396 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.207071 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.207738 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.208747 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.209332 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.209423 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:11.211527 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:11.211531 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:11.211533 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:11.211557 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:11.213085 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.213515 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.213518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.213519 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.213585 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.214079 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.214083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.214085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.214319 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:11.215323 (+ 1004us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:11.215402 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 15:13:11.215506 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 15:13:11.215895 2815 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59147afdeb8941d19683423bd5cd8df3" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } } peers { permanent_uuid: "4b65033630dc4fd2bdc79068b3517cb3" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } } peers { permanent_uuid: "7aba1fd582d640ab8e481cd4b83d9413" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } }
I20260811 15:13:11.216228 2815 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b65033630dc4fd2bdc79068b3517cb3, State: Initialized, Role: FOLLOWER
I20260811 15:13:11.216653 2815 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59147afdeb8941d19683423bd5cd8df3" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } } peers { permanent_uuid: "4b65033630dc4fd2bdc79068b3517cb3" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } } peers { permanent_uuid: "7aba1fd582d640ab8e481cd4b83d9413" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } }
I20260811 15:13:11.218014 2825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.204527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36524 (local address 127.30.8.61:36145)
0811 15:13:11.204652 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.204656 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.204912 (+ 256us) server_negotiation.cc:407] Connection header received
0811 15:13:11.205129 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.205133 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.205194 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.205289 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.206478 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.207248 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.208374 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.208696 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.208766 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:11.213524 (+ 4758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:11.213528 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:11.213531 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:11.213569 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:11.215219 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.216371 (+ 1152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.216374 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.216376 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.216473 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.216929 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.216933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.216934 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.217160 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:11.217330 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:11.217735 (+ 405us) server_negotiation.cc:299] Negotiation successful
0811 15:13:11.217865 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 15:13:11.218397 2815 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:13:11.218976 2683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.209345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:43643 (local address 127.0.0.1:39838)
0811 15:13:11.209653 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:11.209669 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 15:13:11.209754 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:13:11.210230 (+ 476us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:13:11.210233 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:13:11.210423 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:13:11.210532 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:11.210552 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.211580 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.211584 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:11.212346 (+ 762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:11.212360 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.212996 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.213000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:11.213086 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:11.213866 (+ 780us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:13:11.213881 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:13:11.215339 (+ 1458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:13:11.217329 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:11.217332 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:11.217334 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:11.217587 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:11.217912 (+ 325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:11.217916 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:11.217918 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:11.218008 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:11.218442 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:13:11.218446 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:13:11.218566 (+ 120us) client_negotiation.cc:769] Sending connection context
0811 15:13:11.218651 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 15:13:11.218795 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218}
I20260811 15:13:11.219235 2953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413: Bootstrap starting.
I20260811 15:13:11.220002 2982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.209600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39838 (local address 127.30.8.60:43643)
0811 15:13:11.209978 (+ 378us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.209987 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.210001 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:13:11.210048 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.210051 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.210092 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.210185 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.210678 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.211436 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.212507 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.212861 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.213313 (+ 452us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:11.215493 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:11.215497 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:11.215499 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:11.215523 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:11.217204 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.217714 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.217717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.217718 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.217805 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.218123 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.218126 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.218127 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.218325 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:11.219612 (+ 1287us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:11.219690 (+ 78us) server_negotiation.cc:299] Negotiation successful
0811 15:13:11.219842 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":309,"thread_start_us":128,"threads_started":1}
I20260811 15:13:11.220614 2983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59147afdeb8941d19683423bd5cd8df3" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } } peers { permanent_uuid: "4b65033630dc4fd2bdc79068b3517cb3" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } } peers { permanent_uuid: "7aba1fd582d640ab8e481cd4b83d9413" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } } }
I20260811 15:13:11.220759 2983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:13:11.222101 2953 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413: Neither blocks nor log segments found. Creating new log.
I20260811 15:13:11.223069 2953 log.cc:824] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413: Log is configured to *not* fsync() on all Append() calls
I20260811 15:13:11.223551 2679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3: Bootstrap starting.
W20260811 15:13:11.225199 3003 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:13:11.225272 3003 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:13:11.225744 2679 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3: Neither blocks nor log segments found. Creating new log.
I20260811 15:13:11.223253 2692 master_runner.cc:186] Error getting master registration for 127.30.8.62:46877: OK, instance_id { permanent_uuid: "59147afdeb8941d19683423bd5cd8df3" instance_seqno: 1786461190676004 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 15:13:11.226420 2679 log.cc:824] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3: Log is configured to *not* fsync() on all Append() calls
I20260811 15:13:11.226547 2692 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 15:13:11.228951 2953 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413: No bootstrap required, opened a new log
I20260811 15:13:11.229483 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:13:11.232125 2679 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3: No bootstrap required, opened a new log
I20260811 15:13:11.231057 2622 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "59147afdeb8941d19683423bd5cd8df3"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54714
I20260811 15:13:11.232913 2953 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59147afdeb8941d19683423bd5cd8df3" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } } peers { permanent_uuid: "4b65033630dc4fd2bdc79068b3517cb3" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } } peers { permanent_uuid: "7aba1fd582d640ab8e481cd4b83d9413" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } }
I20260811 15:13:11.233204 2953 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aba1fd582d640ab8e481cd4b83d9413, State: Initialized, Role: FOLLOWER
I20260811 15:13:11.233559 2953 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59147afdeb8941d19683423bd5cd8df3" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } } peers { permanent_uuid: "4b65033630dc4fd2bdc79068b3517cb3" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } } peers { permanent_uuid: "7aba1fd582d640ab8e481cd4b83d9413" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } }
I20260811 15:13:11.234249 3008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59147afdeb8941d19683423bd5cd8df3" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } } peers { permanent_uuid: "4b65033630dc4fd2bdc79068b3517cb3" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } } peers { permanent_uuid: "7aba1fd582d640ab8e481cd4b83d9413" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } } }
I20260811 15:13:11.234582 3008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:13:11.235179 2953 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:13:11.235463 2679 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59147afdeb8941d19683423bd5cd8df3" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } } peers { permanent_uuid: "4b65033630dc4fd2bdc79068b3517cb3" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } } peers { permanent_uuid: "7aba1fd582d640ab8e481cd4b83d9413" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } }
I20260811 15:13:11.235728 2679 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59147afdeb8941d19683423bd5cd8df3, State: Initialized, Role: FOLLOWER
I20260811 15:13:11.236076 2679 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59147afdeb8941d19683423bd5cd8df3" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } } peers { permanent_uuid: "4b65033630dc4fd2bdc79068b3517cb3" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } } peers { permanent_uuid: "7aba1fd582d640ab8e481cd4b83d9413" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } }
I20260811 15:13:11.236662 3021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59147afdeb8941d19683423bd5cd8df3" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } } peers { permanent_uuid: "4b65033630dc4fd2bdc79068b3517cb3" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } } peers { permanent_uuid: "7aba1fd582d640ab8e481cd4b83d9413" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } } }
I20260811 15:13:11.236943 3021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:13:11.237587 2679 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:13:11.237661 2622 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 15:13:11.237778 2622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:13:11.239223 2622 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59147afdeb8941d19683423bd5cd8df3" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } } peers { permanent_uuid: "4b65033630dc4fd2bdc79068b3517cb3" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } } peers { permanent_uuid: "7aba1fd582d640ab8e481cd4b83d9413" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } }
I20260811 15:13:11.240274 2622 leader_election.cc:288] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [CANDIDATE]: Term 1 election: Requested vote from peers 4b65033630dc4fd2bdc79068b3517cb3 (127.30.8.61:36145), 7aba1fd582d640ab8e481cd4b83d9413 (127.30.8.60:43643)
I20260811 15:13:11.241734 2769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "59147afdeb8941d19683423bd5cd8df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4b65033630dc4fd2bdc79068b3517cb3"
I20260811 15:13:11.241818 2906 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "59147afdeb8941d19683423bd5cd8df3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7aba1fd582d640ab8e481cd4b83d9413"
I20260811 15:13:11.242134 2906 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:13:11.242164 2769 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:13:11.239553 2826 master_runner.cc:186] Error getting master registration for 127.30.8.62:46877: OK, instance_id { permanent_uuid: "59147afdeb8941d19683423bd5cd8df3" instance_seqno: 1786461190676004 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 15:13:11.243868 2906 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59147afdeb8941d19683423bd5cd8df3 in term 1.
I20260811 15:13:11.243927 2826 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 15:13:11.244053 2769 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59147afdeb8941d19683423bd5cd8df3 in term 1.
I20260811 15:13:11.244468 2581 leader_election.cc:302] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59147afdeb8941d19683423bd5cd8df3, 7aba1fd582d640ab8e481cd4b83d9413; no voters:
I20260811 15:13:11.244915 3021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:13:11.245172 3021 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [term 1 LEADER]: Becoming Leader. State: Replica: 59147afdeb8941d19683423bd5cd8df3, State: Running, Role: LEADER
I20260811 15:13:11.245607 3021 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59147afdeb8941d19683423bd5cd8df3" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } } peers { permanent_uuid: "4b65033630dc4fd2bdc79068b3517cb3" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } } peers { permanent_uuid: "7aba1fd582d640ab8e481cd4b83d9413" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } }
W20260811 15:13:11.248379 3029 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:13:11.248489 3029 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:13:11.251093 3060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59147afdeb8941d19683423bd5cd8df3. Latest consensus state: current_term: 1 leader_uuid: "59147afdeb8941d19683423bd5cd8df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59147afdeb8941d19683423bd5cd8df3" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } } peers { permanent_uuid: "4b65033630dc4fd2bdc79068b3517cb3" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } } peers { permanent_uuid: "7aba1fd582d640ab8e481cd4b83d9413" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } } }
I20260811 15:13:11.251267 3060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [sys.catalog]: This master's current role is: LEADER
W20260811 15:13:11.255344 3052 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:13:11.257283 3052 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:13:11.261981 2979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.250834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54726 (local address 127.30.8.62:46877)
0811 15:13:11.251266 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.251270 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.251980 (+ 710us) server_negotiation.cc:407] Connection header received
0811 15:13:11.252160 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.252163 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.252218 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.252269 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.253489 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.254300 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.255603 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.255951 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.256043 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:11.257765 (+ 1722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:11.257768 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:11.257771 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:11.257797 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:11.259574 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.260242 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.260246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.260248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.260339 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.260629 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.260632 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.260634 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.260887 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:11.260990 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:11.261539 (+ 549us) server_negotiation.cc:299] Negotiation successful
0811 15:13:11.261762 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360}
I20260811 15:13:11.264863 2973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.250412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39854 (local address 127.30.8.60:43643)
0811 15:13:11.250551 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.250555 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.251689 (+ 1134us) server_negotiation.cc:407] Connection header received
0811 15:13:11.251800 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.251803 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.251859 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.251961 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.253132 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.253893 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.255105 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.255425 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.255508 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:11.257483 (+ 1975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:11.257488 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:11.257490 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:11.257516 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:11.259180 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.263463 (+ 4283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.263468 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.263470 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.263570 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.263915 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.263918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.263920 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.264160 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:11.264276 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:11.264579 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 15:13:11.264703 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 15:13:11.266279 3068 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:13:11.267129 2825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:11.249715 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36528 (local address 127.30.8.61:36145)
0811 15:13:11.249870 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.249875 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.251360 (+ 1485us) server_negotiation.cc:407] Connection header received
0811 15:13:11.252416 (+ 1056us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.252419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.252583 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.252714 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.258858 (+ 6144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.259686 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.260881 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:11.261233 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:11.261336 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:11.263121 (+ 1785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:11.263125 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:11.263128 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:11.263158 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:11.264820 (+ 1662us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.265339 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.265343 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.265345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.265436 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:11.265760 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:11.265764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:11.265765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:11.265994 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:11.266141 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:11.266781 (+ 640us) server_negotiation.cc:299] Negotiation successful
0811 15:13:11.266908 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 15:13:11.267318 3068 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:13:11.275038 2769 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3 [term 1 FOLLOWER]: Refusing update from remote peer 59147afdeb8941d19683423bd5cd8df3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:13:11.275439 2906 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413 [term 1 FOLLOWER]: Refusing update from remote peer 59147afdeb8941d19683423bd5cd8df3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:13:11.275801 3021 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4b65033630dc4fd2bdc79068b3517cb3" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:13:11.276140 3060 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7aba1fd582d640ab8e481cd4b83d9413" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:13:11.280449 3008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59147afdeb8941d19683423bd5cd8df3. Latest consensus state: current_term: 1 leader_uuid: "59147afdeb8941d19683423bd5cd8df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59147afdeb8941d19683423bd5cd8df3" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } } peers { permanent_uuid: "4b65033630dc4fd2bdc79068b3517cb3" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } } peers { permanent_uuid: "7aba1fd582d640ab8e481cd4b83d9413" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } } }
I20260811 15:13:11.280613 3008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:13:11.281069 2983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59147afdeb8941d19683423bd5cd8df3. Latest consensus state: current_term: 1 leader_uuid: "59147afdeb8941d19683423bd5cd8df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59147afdeb8941d19683423bd5cd8df3" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } } peers { permanent_uuid: "4b65033630dc4fd2bdc79068b3517cb3" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } } peers { permanent_uuid: "7aba1fd582d640ab8e481cd4b83d9413" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } } }
I20260811 15:13:11.281206 2983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:13:11.288898 3021 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "59147afdeb8941d19683423bd5cd8df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59147afdeb8941d19683423bd5cd8df3" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } } peers { permanent_uuid: "4b65033630dc4fd2bdc79068b3517cb3" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } } peers { permanent_uuid: "7aba1fd582d640ab8e481cd4b83d9413" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } } }
I20260811 15:13:11.289086 3021 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [sys.catalog]: This master's current role is: LEADER
I20260811 15:13:11.289378 3070 mvcc.cc:205] Tried to move back new op lower bound from 7317345039452004352 to 7317345039352397824. Current Snapshot: MvccSnapshot[applied={T|T < 7317345039452004352}]
I20260811 15:13:11.290709 3008 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "59147afdeb8941d19683423bd5cd8df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59147afdeb8941d19683423bd5cd8df3" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } } peers { permanent_uuid: "4b65033630dc4fd2bdc79068b3517cb3" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } } peers { permanent_uuid: "7aba1fd582d640ab8e481cd4b83d9413" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } } }
I20260811 15:13:11.290846 3008 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:13:11.292897 3072 mvcc.cc:205] Tried to move back new op lower bound from 7317345039452004352 to 7317345039352397824. Current Snapshot: MvccSnapshot[applied={T|T < 7317345039452004352}]
I20260811 15:13:11.295894 3060 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "59147afdeb8941d19683423bd5cd8df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59147afdeb8941d19683423bd5cd8df3" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } } peers { permanent_uuid: "4b65033630dc4fd2bdc79068b3517cb3" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } } peers { permanent_uuid: "7aba1fd582d640ab8e481cd4b83d9413" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } } }
I20260811 15:13:11.296078 3060 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3 [sys.catalog]: This master's current role is: LEADER
I20260811 15:13:11.297490 2983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "59147afdeb8941d19683423bd5cd8df3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "59147afdeb8941d19683423bd5cd8df3" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 46877 } } peers { permanent_uuid: "4b65033630dc4fd2bdc79068b3517cb3" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 36145 } } peers { permanent_uuid: "7aba1fd582d640ab8e481cd4b83d9413" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 43643 } } }
I20260811 15:13:11.297622 2983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:13:11.302747 3075 mvcc.cc:205] Tried to move back new op lower bound from 7317345039452004352 to 7317345039352397824. Current Snapshot: MvccSnapshot[applied={T|T < 7317345039452004352}]
I20260811 15:13:11.307726 3068 catalog_manager.cc:1347] Generated new cluster ID: a62088af7c844c76a790fa1f97ec89cd
I20260811 15:13:11.307848 3068 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:13:11.340777 3068 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:13:11.342595 3068 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:13:11.365700 3068 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59147afdeb8941d19683423bd5cd8df3: Generated new TSK 0
I20260811 15:13:11.366863 3068 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 15:13:11.408334 2825 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:11.404567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36536 (local address 127.30.8.61:36145)
0811 15:13:11.404748 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.404754 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.406743 (+ 1989us) server_negotiation.cc:407] Connection header received
0811 15:13:11.407086 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.407090 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.407162 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.407226 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.408122 (+ 896us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36536: BlockingRecv error: recv got EOF from 127.0.0.1:36536 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260811 15:13:11.411337 2973 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:11.405913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39862 (local address 127.30.8.60:43643)
0811 15:13:11.406138 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.406144 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.409776 (+ 3632us) server_negotiation.cc:407] Connection header received
0811 15:13:11.410194 (+ 418us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.410199 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.410281 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.410347 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.411139 (+ 792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39862: BlockingRecv error: recv got EOF from 127.0.0.1:39862 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260811 15:13:11.413401 2979 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:11.409459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54738 (local address 127.30.8.62:46877)
0811 15:13:11.409686 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.409692 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.412187 (+ 2495us) server_negotiation.cc:407] Connection header received
0811 15:13:11.412430 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.412434 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.412504 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.412568 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.413222 (+ 654us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:54738: BlockingRecv error: recv got EOF from 127.0.0.1:54738 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260811 15:13:11.420375 2825 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:11.417738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36538 (local address 127.30.8.61:36145)
0811 15:13:11.417910 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.417916 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.418495 (+ 579us) server_negotiation.cc:407] Connection header received
0811 15:13:11.418784 (+ 289us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.418789 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.419096 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.419186 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.420209 (+ 1023us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36538: BlockingRecv error: recv got EOF from 127.0.0.1:36538 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 15:13:11.427150 2825 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:11.425051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36542 (local address 127.30.8.61:36145)
0811 15:13:11.425199 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.425205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.425917 (+ 712us) server_negotiation.cc:407] Connection header received
0811 15:13:11.426167 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.426171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.426237 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.426310 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.426997 (+ 687us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36542: BlockingRecv error: recv got EOF from 127.0.0.1:36542 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 15:13:11.433751 2825 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:11.431918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36546 (local address 127.30.8.61:36145)
0811 15:13:11.432045 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.432050 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.432669 (+ 619us) server_negotiation.cc:407] Connection header received
0811 15:13:11.432873 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.432876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.432935 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.432988 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.433599 (+ 611us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36546: BlockingRecv error: recv got EOF from 127.0.0.1:36546 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 15:13:11.446434 2825 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:11.443706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36548 (local address 127.30.8.61:36145)
0811 15:13:11.443920 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.443926 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.444803 (+ 877us) server_negotiation.cc:407] Connection header received
0811 15:13:11.445102 (+ 299us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.445107 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.445177 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.445237 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.446132 (+ 895us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36548: BlockingRecv error: recv got EOF from 127.0.0.1:36548 (error 108)
Metrics: {"server-negotiator.queue_time_us":112}
W20260811 15:13:11.461033 2825 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:11.459456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36562 (local address 127.30.8.61:36145)
0811 15:13:11.459604 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.459609 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.460104 (+ 495us) server_negotiation.cc:407] Connection header received
0811 15:13:11.460271 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.460274 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.460327 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.460399 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.460885 (+ 486us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36562: BlockingRecv error: recv got EOF from 127.0.0.1:36562 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 15:13:11.480247 2825 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:11.479119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36576 (local address 127.30.8.61:36145)
0811 15:13:11.479265 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.479271 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.479363 (+ 92us) server_negotiation.cc:407] Connection header received
0811 15:13:11.479487 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.479491 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.479550 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.479657 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.480108 (+ 451us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36576: BlockingRecv error: recv got EOF from 127.0.0.1:36576 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260811 15:13:11.515749 2825 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:11.514623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36584 (local address 127.30.8.61:36145)
0811 15:13:11.514772 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.514776 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.514960 (+ 184us) server_negotiation.cc:407] Connection header received
0811 15:13:11.515085 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.515088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.515139 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.515210 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.515621 (+ 411us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36584: BlockingRecv error: recv got EOF from 127.0.0.1:36584 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 15:13:11.583199 2825 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:11.582095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36596 (local address 127.30.8.61:36145)
0811 15:13:11.582302 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.582307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.582326 (+ 19us) server_negotiation.cc:407] Connection header received
0811 15:13:11.582416 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.582419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.582468 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.582537 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.583084 (+ 547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36596: BlockingRecv error: recv got EOF from 127.0.0.1:36596 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 15:13:11.714689 2825 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:11.713468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36608 (local address 127.30.8.61:36145)
0811 15:13:11.713627 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.713632 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.713758 (+ 126us) server_negotiation.cc:407] Connection header received
0811 15:13:11.713923 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.713926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.713979 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.714068 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.714574 (+ 506us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36608: BlockingRecv error: recv got EOF from 127.0.0.1:36608 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 15:13:11.974800 2825 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:11.973475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36618 (local address 127.30.8.61:36145)
0811 15:13:11.973657 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:11.973662 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:11.973681 (+ 19us) server_negotiation.cc:407] Connection header received
0811 15:13:11.973908 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:11.973912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:11.973970 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:11.974051 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:11.974660 (+ 609us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36618: BlockingRecv error: recv got EOF from 127.0.0.1:36618 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 15:13:12.226692 3003 catalog_manager.cc:1259] Loaded cluster ID: a62088af7c844c76a790fa1f97ec89cd
I20260811 15:13:12.226790 3003 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3: loading cluster ID for follower catalog manager: success
I20260811 15:13:12.228566 3003 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3: acquiring CA information for follower catalog manager: success
I20260811 15:13:12.229831 3003 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4b65033630dc4fd2bdc79068b3517cb3: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 15:13:12.250010 3029 catalog_manager.cc:1259] Loaded cluster ID: a62088af7c844c76a790fa1f97ec89cd
I20260811 15:13:12.250110 3029 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413: loading cluster ID for follower catalog manager: success
I20260811 15:13:12.251861 3029 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413: acquiring CA information for follower catalog manager: success
I20260811 15:13:12.253103 3029 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7aba1fd582d640ab8e481cd4b83d9413: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 15:13:12.493876 3197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:12.489624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36628 (local address 127.30.8.61:36145)
0811 15:13:12.489936 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:12.489940 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:12.489961 (+ 21us) server_negotiation.cc:407] Connection header received
0811 15:13:12.490251 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:12.490255 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:12.490527 (+ 272us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:12.490661 (+ 134us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:13:12.490903 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:12.491493 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:12.492470 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:12.492684 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:12.492754 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:12.493030 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:13:12.493525 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:13:12.493618 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:12.493673 (+ 55us) server_negotiation.cc:299] Negotiation successful
0811 15:13:12.493729 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":234,"thread_start_us":100,"threads_started":1}
I20260811 15:13:12.500622 3211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:12.497182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39870 (local address 127.30.8.60:43643)
0811 15:13:12.497391 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:12.497395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:12.497409 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:13:12.497500 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:12.497503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:12.497634 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:12.497735 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:13:12.497893 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:12.498418 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:12.499243 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:12.499462 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:12.499536 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:12.499796 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:13:12.500277 (+ 481us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:13:12.500372 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:12.500398 (+ 26us) server_negotiation.cc:299] Negotiation successful
0811 15:13:12.500452 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":68,"threads_started":1}
I20260811 15:13:12.508036 3211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:12.504057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39878 (local address 127.30.8.60:43643)
0811 15:13:12.504181 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:12.504186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:12.504261 (+ 75us) server_negotiation.cc:407] Connection header received
0811 15:13:12.504407 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:12.504410 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:12.504462 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:12.504528 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:13:12.504756 (+ 228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:12.505492 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:12.506936 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:12.507252 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:12.507344 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:12.507590 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:13:12.507666 (+ 76us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:13:12.507722 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:12.507832 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 15:13:12.507890 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 15:13:12.508342 3197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:12.503727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36640 (local address 127.30.8.61:36145)
0811 15:13:12.503863 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:12.503868 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:12.504339 (+ 471us) server_negotiation.cc:407] Connection header received
0811 15:13:12.504494 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:12.504498 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:12.504560 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:12.504623 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:13:12.504982 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:12.505747 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:12.506970 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:12.507253 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:12.507358 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:12.507714 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:13:12.507786 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:13:12.507831 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:12.508157 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 15:13:12.508215 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 15:13:12.508368 3224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:12.503743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54740 (local address 127.30.8.62:46877)
0811 15:13:12.503976 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:12.503981 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:12.504027 (+ 46us) server_negotiation.cc:407] Connection header received
0811 15:13:12.504290 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:12.504294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:12.504482 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:12.504604 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:13:12.505013 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:12.505599 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:12.506647 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:12.506972 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:12.507071 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:12.507397 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:13:12.507814 (+ 417us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:13:12.507898 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:12.508177 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 15:13:12.508238 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":72,"threads_started":1}
I20260811 15:13:12.517570 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 2552
I20260811 15:13:12.537367 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 2692
I20260811 15:13:12.557057 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 2826
2026-08-11T15:13:12Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3446 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 15:13:12 dist-test-slave-1vmg krb5kdc[3231](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 15:13:12 dist-test-slave-1vmg krb5kdc[3231](info): set up 2 sockets
Aug 11 15:13:12 dist-test-slave-1vmg krb5kdc[3231](info): commencing operation
krb5kdc: starting...
W20260811 15:13:13.773537 30752 mini_kdc.cc:121] Time spent starting KDC: real 1.183s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/test-user.keytab.
Aug 11 15:13:13 dist-test-slave-1vmg krb5kdc[3231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T15:13:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T15:13:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.8.62@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.62@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
Entry for principal kudu/127.30.8.62 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab.
I20260811 15:13:13.926851 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:44883
--webserver_interface=127.30.8.62
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:46469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:44883,127.30.8.61:43357,127.30.8.60:37557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 15:13:14.051779 3252 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 15:13:14.052114 3252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:13:14.052140 3252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:13:14.052203 3252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:13:14.056815 3252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:13:14.056900 3252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:13:14.056919 3252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:13:14.056951 3252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:13:14.056982 3252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:13:14.062700 3252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:46469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:44883,127.30.8.61:43357,127.30.8.60:37557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.62.keytab
--principal=kudu/127.30.8.62
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.62:44883
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.8.62
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:13:14.063971 3252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:13:14.065001 3252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:13:14.074240 3259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:14.074435 3258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:13:14.074672 3252 server_base.cc:1034] running on GCE node
W20260811 15:13:14.074749 3262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:13:14.075420 3252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:13:14.076704 3252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:13:14.077872 3252 hybrid_clock.cc:648] HybridClock initialized: now 1786461194077854 us; error 33 us; skew 500 ppm
Aug 11 15:13:14 dist-test-slave-1vmg krb5kdc[3231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461194, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:13:14.081028 3252 init.cc:377] Logged in from keytab as kudu/127.30.8.62@KRBTEST.COM (short username kudu)
I20260811 15:13:14.082135 3252 webserver.cc:460] Webserver started at http://127.30.8.62:37863/ using document root <none> and password file <none>
I20260811 15:13:14.082757 3252 fs_manager.cc:359] Metadata directory not provided
I20260811 15:13:14.082834 3252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:13:14.083097 3252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:13:14.085310 3252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-0/data/instance:
uuid: "d6a5f051de274ab98b4bb5be1055769d"
format_stamp: "Formatted at 2026-08-11 15:13:14 on dist-test-slave-1vmg"
server_key: "082b0dff865831ea2df1482555e98c88"
server_key_iv: "dfd0c3c8055d75235c0351b55054e241"
server_key_version: "encryptionkey@0"
I20260811 15:13:14.085934 3252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-0/wal/instance:
uuid: "d6a5f051de274ab98b4bb5be1055769d"
format_stamp: "Formatted at 2026-08-11 15:13:14 on dist-test-slave-1vmg"
server_key: "082b0dff865831ea2df1482555e98c88"
server_key_iv: "dfd0c3c8055d75235c0351b55054e241"
server_key_version: "encryptionkey@0"
I20260811 15:13:14.090042 3252 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 15:13:14.092720 3270 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:13:14.094020 3252 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 15:13:14.094156 3252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-0/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-0/wal
uuid: "d6a5f051de274ab98b4bb5be1055769d"
format_stamp: "Formatted at 2026-08-11 15:13:14 on dist-test-slave-1vmg"
server_key: "082b0dff865831ea2df1482555e98c88"
server_key_iv: "dfd0c3c8055d75235c0351b55054e241"
server_key_version: "encryptionkey@0"
I20260811 15:13:14.094267 3252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:13:14.112882 3252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:13:14.116304 3252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:13:14.116511 3252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:13:14.124003 3252 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.62:44883
I20260811 15:13:14.124329 3356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.62:44883 every 8 connection(s)
I20260811 15:13:14.125104 3252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-0/data/info.pb
I20260811 15:13:14.126866 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 3252
I20260811 15:13:14.127022 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-0/wal/instance
I20260811 15:13:14.127384 30752 external_mini_cluster.cc:1428] Setting key 220127d5ac721bc007db620f7fc3a6a2
I20260811 15:13:14.128778 3357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:13:14.133190 3357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.8.61@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.61@KRBTEST.COM" created.
Aug 11 15:13:14 dist-test-slave-1vmg krb5kdc[3231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461194, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:13:14.149356 3361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.135801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52582 (local address 127.30.8.62:44883)
0811 15:13:14.136342 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.136362 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.136645 (+ 283us) server_negotiation.cc:407] Connection header received
0811 15:13:14.137735 (+ 1090us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.137740 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.137943 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.138136 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.139146 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.140045 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.141193 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.141579 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.141715 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.144465 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:14.144471 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:14.144484 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:14.144507 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:14.146074 (+ 1567us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.146540 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.146546 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.146547 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.146660 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.146989 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.146993 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.146994 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.147436 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:14.147565 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:14.147963 (+ 398us) server_negotiation.cc:299] Negotiation successful
0811 15:13:14.148216 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":372,"thread_start_us":162,"threads_started":1}
I20260811 15:13:14.149370 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.135801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:44883 (local address 127.0.0.1:52582)
0811 15:13:14.136472 (+ 671us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.136520 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0811 15:13:14.137446 (+ 926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:13:14.138160 (+ 714us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:13:14.138165 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:13:14.138578 (+ 413us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:13:14.138969 (+ 391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:14.138991 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.140183 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.140186 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:14.141058 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:14.141071 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.141750 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.141754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:14.141937 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.142542 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:13:14.142563 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:13:14.144311 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:13:14.146184 (+ 1873us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:14.146193 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:14.146206 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:14.146432 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:14.146749 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:14.146752 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:14.146754 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:14.146870 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:14.147557 (+ 687us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:13:14.147564 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:13:14.147708 (+ 144us) client_negotiation.cc:769] Sending connection context
0811 15:13:14.147864 (+ 156us) client_negotiation.cc:240] Negotiation successful
0811 15:13:14.148277 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":139,"spinlock_wait_cycles":60288,"thread_start_us":183,"threads_started":1}
I20260811 15:13:14.152274 3357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } has no permanent_uuid. Determining permanent_uuid...
I20260811 15:13:14.153100 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.152779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:43357 (local address 127.0.0.1:35412)
0811 15:13:14.152938 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.153014 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.61:43357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260811 15:13:14.153304 3275 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.8.61:43357: connect: Connection refused (error 111)
I20260811 15:13:14.154201 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.153985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:43357 (local address 127.0.0.1:35426)
0811 15:13:14.154083 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.154136 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.61:43357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260811 15:13:14.154325 3357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.61:43357: Network error: Client connection negotiation failed: client connection to 127.30.8.61:43357: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.61 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.61.keytab.
Entry for principal kudu/127.30.8.61 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.61.keytab.
I20260811 15:13:14.158052 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.61:43357
--webserver_interface=127.30.8.61
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.61.keytab
--principal=kudu/127.30.8.61
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:46469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:44883,127.30.8.61:43357,127.30.8.60:37557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:13:14.203574 3357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } attempt: 1
I20260811 15:13:14.204847 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.204227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:43357 (local address 127.0.0.1:35436)
0811 15:13:14.204572 (+ 345us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.204706 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.61:43357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
I20260811 15:13:14.205750 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.205517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:43357 (local address 127.0.0.1:35442)
0811 15:13:14.205627 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.205682 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.61:43357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 15:13:14.205946 3357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.61:43357: Network error: Client connection negotiation failed: client connection to 127.30.8.61:43357: connect: Connection refused (error 111)
I20260811 15:13:14.274219 3357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } attempt: 2
I20260811 15:13:14.275197 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.274838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:43357 (local address 127.0.0.1:35458)
0811 15:13:14.275038 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.275104 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.61:43357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
I20260811 15:13:14.276003 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.275786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:43357 (local address 127.0.0.1:35474)
0811 15:13:14.275894 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.275945 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.61:43357: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 15:13:14.276119 3357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.61:43357: Network error: Client connection negotiation failed: client connection to 127.30.8.61:43357: connect: Connection refused (error 111)
W20260811 15:13:14.280378 3370 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 15:13:14.280712 3370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:13:14.280757 3370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:13:14.280822 3370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:13:14.285080 3370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:13:14.285164 3370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:13:14.285192 3370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:13:14.285215 3370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:13:14.285245 3370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:13:14.291064 3370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:46469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:44883,127.30.8.61:43357,127.30.8.60:37557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.61.keytab
--principal=kudu/127.30.8.61
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.61:43357
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.8.61
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:13:14.292316 3370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:13:14.293416 3370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:13:14.302479 3379 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:14.302479 3382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:13:14.302652 3370 server_base.cc:1034] running on GCE node
W20260811 15:13:14.302500 3377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:13:14.303443 3370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:13:14.304668 3370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:13:14.305845 3370 hybrid_clock.cc:648] HybridClock initialized: now 1786461194305820 us; error 43 us; skew 500 ppm
Aug 11 15:13:14 dist-test-slave-1vmg krb5kdc[3231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461194, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.61@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:13:14.308746 3370 init.cc:377] Logged in from keytab as kudu/127.30.8.61@KRBTEST.COM (short username kudu)
I20260811 15:13:14.309728 3370 webserver.cc:460] Webserver started at http://127.30.8.61:37527/ using document root <none> and password file <none>
I20260811 15:13:14.310305 3370 fs_manager.cc:359] Metadata directory not provided
I20260811 15:13:14.310367 3370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:13:14.310560 3370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:13:14.312786 3370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-1/data/instance:
uuid: "56c6940e484e48e19c8680465145ecbc"
format_stamp: "Formatted at 2026-08-11 15:13:14 on dist-test-slave-1vmg"
server_key: "2a0cdf7a8cfa2e3e0a7656cb753d6165"
server_key_iv: "804c3f976b81a30132c17483d1562446"
server_key_version: "encryptionkey@0"
I20260811 15:13:14.313349 3370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-1/wal/instance:
uuid: "56c6940e484e48e19c8680465145ecbc"
format_stamp: "Formatted at 2026-08-11 15:13:14 on dist-test-slave-1vmg"
server_key: "2a0cdf7a8cfa2e3e0a7656cb753d6165"
server_key_iv: "804c3f976b81a30132c17483d1562446"
server_key_version: "encryptionkey@0"
I20260811 15:13:14.317435 3370 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 15:13:14.320025 3393 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:13:14.321342 3370 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 15:13:14.321466 3370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-1/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-1/wal
uuid: "56c6940e484e48e19c8680465145ecbc"
format_stamp: "Formatted at 2026-08-11 15:13:14 on dist-test-slave-1vmg"
server_key: "2a0cdf7a8cfa2e3e0a7656cb753d6165"
server_key_iv: "804c3f976b81a30132c17483d1562446"
server_key_version: "encryptionkey@0"
I20260811 15:13:14.321571 3370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:13:14.332551 3370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:13:14.335973 3370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:13:14.336163 3370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:13:14.343598 3370 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.61:43357
I20260811 15:13:14.343878 3480 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.61:43357 every 8 connection(s)
I20260811 15:13:14.344666 3370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-1/data/info.pb
I20260811 15:13:14.347280 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 3370
I20260811 15:13:14.347411 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-1/wal/instance
I20260811 15:13:14.347736 30752 external_mini_cluster.cc:1428] Setting key 0026f550a6d00414205c7ce15f174b4f
I20260811 15:13:14.347939 3481 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:13:14.352569 3481 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.8.60@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.8.60@KRBTEST.COM" created.
Aug 11 15:13:14 dist-test-slave-1vmg krb5kdc[3231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461194, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.61@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:13:14.367103 3361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.355115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52592 (local address 127.30.8.62:44883)
0811 15:13:14.355300 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.355305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.356219 (+ 914us) server_negotiation.cc:407] Connection header received
0811 15:13:14.357234 (+ 1015us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.357238 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.357307 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.357402 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.358673 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.359316 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.360576 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.360822 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.360917 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.363501 (+ 2584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:14.363504 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:14.363506 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:14.363541 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:14.364892 (+ 1351us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.365424 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.365427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.365428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.365508 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.365851 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.365855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.365856 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.366059 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:14.366149 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:14.366740 (+ 591us) server_negotiation.cc:299] Negotiation successful
0811 15:13:14.366883 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 15:13:14.367306 3357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } attempt: 3
I20260811 15:13:14.368212 3486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.355492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:44883 (local address 127.0.0.1:52592)
0811 15:13:14.356026 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.356100 (+ 74us) client_negotiation.cc:174] Beginning negotiation
0811 15:13:14.357000 (+ 900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:13:14.357589 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:13:14.357596 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:13:14.357949 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:13:14.358523 (+ 574us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:14.358539 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.359449 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.359452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:14.360440 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:14.360451 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.360978 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.360982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:14.361168 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.361768 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:13:14.361785 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:13:14.363320 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:13:14.365012 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:14.365019 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:14.365048 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:14.365325 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:14.365602 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:14.365605 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:14.365607 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:14.365761 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:14.366164 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:13:14.366171 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:13:14.366466 (+ 295us) client_negotiation.cc:769] Sending connection context
0811 15:13:14.366660 (+ 194us) client_negotiation.cc:240] Negotiation successful
0811 15:13:14.367076 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":131,"threads_started":1}
I20260811 15:13:14.369658 3481 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 15:13:14 dist-test-slave-1vmg krb5kdc[3231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461194, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.61@KRBTEST.COM for kudu/127.30.8.61@KRBTEST.COM
Aug 11 15:13:14 dist-test-slave-1vmg krb5kdc[3231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461194, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for kudu/127.30.8.61@KRBTEST.COM
I20260811 15:13:14.381059 3486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.370367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:43357 (local address 127.0.0.1:35484)
0811 15:13:14.370567 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.370584 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 15:13:14.370666 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:13:14.371085 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:13:14.371088 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:13:14.371266 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:13:14.371355 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:14.371360 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.372415 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.372419 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:14.372975 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:14.372984 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.373706 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.373710 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:14.373848 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.374384 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:13:14.374400 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:13:14.375773 (+ 1373us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:13:14.378801 (+ 3028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:14.378804 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:14.378806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:14.379090 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:14.379663 (+ 573us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:14.379666 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:14.379668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:14.379786 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:14.380453 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:13:14.380456 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:13:14.380585 (+ 129us) client_negotiation.cc:769] Sending connection context
0811 15:13:14.380701 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 15:13:14.380867 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"mutex_wait_us":2,"spinlock_wait_cycles":1920}
I20260811 15:13:14.381348 3496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.370414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35484 (local address 127.30.8.61:43357)
0811 15:13:14.370861 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.370868 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.370876 (+ 8us) server_negotiation.cc:407] Connection header received
0811 15:13:14.370900 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.370902 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.370976 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.371057 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.371457 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.372260 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.373092 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.373556 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.373655 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.375928 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:14.375941 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:14.375957 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:14.375984 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:14.378633 (+ 2649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.379340 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.379343 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.379345 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.379521 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.379892 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.379896 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.379898 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.380340 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:14.380490 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:14.380830 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 15:13:14.380988 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":395,"spinlock_wait_cycles":1280,"thread_start_us":205,"threads_started":1}
I20260811 15:13:14.381606 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.367736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:43357 (local address 127.0.0.1:35482)
0811 15:13:14.367927 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.367942 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 15:13:14.368017 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:13:14.369789 (+ 1772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:13:14.369793 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:13:14.369978 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:13:14.370090 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:14.370095 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.371708 (+ 1613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.371711 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:14.372918 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:14.372930 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.373732 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.373737 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:14.373856 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.374456 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:13:14.374471 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:13:14.376248 (+ 1777us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:13:14.378768 (+ 2520us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:14.378772 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:14.378774 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:14.379078 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:14.379663 (+ 585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:14.379666 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:14.379668 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:14.379786 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:14.380908 (+ 1122us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:13:14.380913 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:13:14.381048 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 15:13:14.381174 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 15:13:14.381403 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.8.60 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.60.keytab.
Entry for principal kudu/127.30.8.60 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.60.keytab.
I20260811 15:13:14.382967 3489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.368183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35482 (local address 127.30.8.61:43357)
0811 15:13:14.368998 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.369016 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.369057 (+ 41us) server_negotiation.cc:407] Connection header received
0811 15:13:14.369143 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.369152 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.369499 (+ 347us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.369875 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.370224 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.371541 (+ 1317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.373084 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.373550 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.373690 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.376388 (+ 2698us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:14.376392 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:14.376395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:14.376420 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:14.378650 (+ 2230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.379303 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.379314 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.379316 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.379521 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.380524 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.380526 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.380528 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.380777 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:14.380875 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:14.382544 (+ 1669us) server_negotiation.cc:299] Negotiation successful
0811 15:13:14.382717 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":109,"server-negotiator.queue_time_us":638,"spinlock_wait_cycles":4480,"thread_start_us":122,"threads_started":1}
I20260811 15:13:14.383297 30752 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
/tmp/dist-test-taskxgyPSN/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.60:37557
--webserver_interface=127.30.8.60
--webserver_port=0
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.60.keytab
--principal=kudu/127.30.8.60
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.8.20:46469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.8.62:44883,127.30.8.61:43357,127.30.8.60:37557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 15:13:14.383456 3481 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } has no permanent_uuid. Determining permanent_uuid...
I20260811 15:13:14.384174 3357 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } has no permanent_uuid. Determining permanent_uuid...
I20260811 15:13:14.384310 3486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.384006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37557 (local address 127.0.0.1:40304)
0811 15:13:14.384137 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.384226 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 15:13:14.384634 3398 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
I20260811 15:13:14.384999 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.384638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37557 (local address 127.0.0.1:40308)
0811 15:13:14.384825 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.384906 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 15:13:14.385808 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.385461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37557 (local address 127.0.0.1:40320)
0811 15:13:14.385657 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.385722 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260811 15:13:14.385982 3486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.385727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37557 (local address 127.0.0.1:40336)
0811 15:13:14.385870 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.385916 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260811 15:13:14.386169 3357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.60:37557: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
W20260811 15:13:14.386263 3481 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.60:37557: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
I20260811 15:13:14.417407 3357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } attempt: 1
I20260811 15:13:14.418470 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.418113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37557 (local address 127.0.0.1:40342)
0811 15:13:14.418297 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.418370 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260811 15:13:14.419304 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.419081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37557 (local address 127.0.0.1:40350)
0811 15:13:14.419191 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.419244 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 15:13:14.419433 3357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.60:37557: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
I20260811 15:13:14.435551 3481 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } attempt: 1
I20260811 15:13:14.436673 3486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.436330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37557 (local address 127.0.0.1:40354)
0811 15:13:14.436515 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.436586 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 15:13:14.437644 3486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.437409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37557 (local address 127.0.0.1:40362)
0811 15:13:14.437537 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.437589 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260811 15:13:14.437796 3481 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.60:37557: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
I20260811 15:13:14.494606 3357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } attempt: 2
I20260811 15:13:14.495548 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.495233 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37557 (local address 127.0.0.1:40366)
0811 15:13:14.495386 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.495450 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 15:13:14.496217 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.496011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37557 (local address 127.0.0.1:40380)
0811 15:13:14.496119 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.496163 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 15:13:14.496327 3357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.60:37557: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
I20260811 15:13:14.505962 3481 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } attempt: 2
I20260811 15:13:14.506819 3486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.506544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37557 (local address 127.0.0.1:40386)
0811 15:13:14.506688 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.506744 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260811 15:13:14.507503 3486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.507307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37557 (local address 127.0.0.1:40388)
0811 15:13:14.507406 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.507449 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260811 15:13:14.507616 3481 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.60:37557: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
W20260811 15:13:14.511291 3501 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 15:13:14.511612 3501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 15:13:14.511656 3501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 15:13:14.511719 3501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 15:13:14.515889 3501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 15:13:14.515964 3501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 15:13:14.515990 3501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 15:13:14.516023 3501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 15:13:14.516047 3501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 15:13:14.521790 3501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.8.20:46469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.8.62:44883,127.30.8.61:43357,127.30.8.60:37557
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/krb5kdc/kudu_127.30.8.60.keytab
--principal=kudu/127.30.8.60
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.8.60:37557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.8.60
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 13:59:13 UTC on ccycloud.quasar-sqcett.root.comops.site
build id 4580140
I20260811 15:13:14.523097 3501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 15:13:14.524201 3501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 15:13:14.533341 3514 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:14.533327 3515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 15:13:14.533327 3518 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 15:13:14.533478 3501 server_base.cc:1034] running on GCE node
I20260811 15:13:14.534207 3501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 15:13:14.535442 3501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 15:13:14.536602 3501 hybrid_clock.cc:648] HybridClock initialized: now 1786461194536577 us; error 41 us; skew 500 ppm
Aug 11 15:13:14 dist-test-slave-1vmg krb5kdc[3231](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461194, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.60@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 15:13:14.539453 3501 init.cc:377] Logged in from keytab as kudu/127.30.8.60@KRBTEST.COM (short username kudu)
I20260811 15:13:14.540445 3501 webserver.cc:460] Webserver started at http://127.30.8.60:33895/ using document root <none> and password file <none>
I20260811 15:13:14.541045 3501 fs_manager.cc:359] Metadata directory not provided
I20260811 15:13:14.541116 3501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 15:13:14.541337 3501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 15:13:14.543569 3501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-2/data/instance:
uuid: "b3673b1715fd496d8fa081c853098900"
format_stamp: "Formatted at 2026-08-11 15:13:14 on dist-test-slave-1vmg"
server_key: "10e1ecb19b8397bfe5c0f29a247804a8"
server_key_iv: "d97393e0005d82363e9b0a362126f0dd"
server_key_version: "encryptionkey@0"
I20260811 15:13:14.544137 3501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-2/wal/instance:
uuid: "b3673b1715fd496d8fa081c853098900"
format_stamp: "Formatted at 2026-08-11 15:13:14 on dist-test-slave-1vmg"
server_key: "10e1ecb19b8397bfe5c0f29a247804a8"
server_key_iv: "d97393e0005d82363e9b0a362126f0dd"
server_key_version: "encryptionkey@0"
I20260811 15:13:14.548089 3501 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 15:13:14.550627 3530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 15:13:14.551942 3501 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.001s
I20260811 15:13:14.552069 3501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-2/data,/tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-2/wal
uuid: "b3673b1715fd496d8fa081c853098900"
format_stamp: "Formatted at 2026-08-11 15:13:14 on dist-test-slave-1vmg"
server_key: "10e1ecb19b8397bfe5c0f29a247804a8"
server_key_iv: "d97393e0005d82363e9b0a362126f0dd"
server_key_version: "encryptionkey@0"
I20260811 15:13:14.552176 3501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 15:13:14.593685 3501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 15:13:14.595504 3357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } attempt: 3
I20260811 15:13:14.596534 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.596179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37557 (local address 127.0.0.1:36366)
0811 15:13:14.596369 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.596444 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 15:13:14.597431 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.597213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37557 (local address 127.0.0.1:36370)
0811 15:13:14.597329 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.597376 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 15:13:14.597555 3357 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.60:37557: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
I20260811 15:13:14.598263 3501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 15:13:14.598505 3501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 15:13:14.598752 3481 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } attempt: 3
I20260811 15:13:14.599483 3486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.599247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37557 (local address 127.0.0.1:36376)
0811 15:13:14.599370 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.599420 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260811 15:13:14.600473 3486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.600182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37557 (local address 127.0.0.1:36386)
0811 15:13:14.600372 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.600424 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122}
W20260811 15:13:14.600611 3481 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.8.60:37557: Network error: Client connection negotiation failed: client connection to 127.30.8.60:37557: connect: Connection refused (error 111)
I20260811 15:13:14.607990 3501 rpc_server.cc:307] RPC server started. Bound to: 127.30.8.60:37557
I20260811 15:13:14.608338 3620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.8.60:37557 every 8 connection(s)
I20260811 15:13:14.609336 3501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-2/data/info.pb
I20260811 15:13:14.612742 3626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 15:13:14.613201 30752 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu as pid 3501
I20260811 15:13:14.613323 30752 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskxgyPSN/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786460893564915-30752-0/minicluster-data/master-2/wal/instance
I20260811 15:13:14.613606 30752 external_mini_cluster.cc:1428] Setting key 3acbc69bb1a9bd95cfead8b00e522e82
I20260811 15:13:14.618094 3626 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 15:13:14 dist-test-slave-1vmg krb5kdc[3231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.60@KRBTEST.COM
Aug 11 15:13:14 dist-test-slave-1vmg krb5kdc[3231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461194, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.60@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:13:14.630235 3629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.615055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36398 (local address 127.30.8.60:37557)
0811 15:13:14.615520 (+ 465us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.615536 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.615584 (+ 48us) server_negotiation.cc:407] Connection header received
0811 15:13:14.616278 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.616297 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.616666 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.616968 (+ 302us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.618162 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.619625 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.620939 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.621736 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.621929 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.624706 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:14.624713 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:14.624725 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:14.624750 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:14.626510 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.626947 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.626955 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.626962 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.627112 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.627372 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.627376 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.627378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.627976 (+ 598us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:14.628216 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:14.628848 (+ 632us) server_negotiation.cc:299] Negotiation successful
0811 15:13:14.629183 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"spinlock_wait_cycles":6016,"thread_start_us":151,"threads_started":1}
I20260811 15:13:14.630424 3361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.620251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43994 (local address 127.30.8.62:44883)
0811 15:13:14.620457 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.620461 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.620695 (+ 234us) server_negotiation.cc:407] Connection header received
0811 15:13:14.620869 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.620873 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.620952 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.621031 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.622026 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.622722 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.623671 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.623894 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.623974 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.626554 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:14.626558 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:14.626561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:14.626593 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:14.628099 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.628978 (+ 879us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.628981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.628983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.629073 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.629390 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.629393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.629395 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.629645 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:14.629763 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:14.630152 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 15:13:14.630287 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 15:13:14.630452 3630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.620112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.62:44883 (local address 127.0.0.1:43994)
0811 15:13:14.620538 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.620602 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 15:13:14.620746 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:13:14.621120 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:13:14.621126 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:13:14.621634 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:13:14.621842 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:14.621851 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.622853 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.622856 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:14.623545 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:14.623558 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.624043 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.624057 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:14.624175 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.624681 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:13:14.624700 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:13:14.626391 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:13:14.628261 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:14.628281 (+ 20us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:14.628292 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:14.628852 (+ 560us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:14.629181 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:14.629184 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:14.629187 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:14.629287 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:14.629805 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:13:14.629811 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:13:14.629945 (+ 134us) client_negotiation.cc:769] Sending connection context
0811 15:13:14.630079 (+ 134us) client_negotiation.cc:240] Negotiation successful
0811 15:13:14.630240 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":246,"spinlock_wait_cycles":3840,"thread_start_us":136,"threads_started":1}
I20260811 15:13:14.632743 3626 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 15:13:14 dist-test-slave-1vmg krb5kdc[3231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461194, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.60@KRBTEST.COM for kudu/127.30.8.61@KRBTEST.COM
Aug 11 15:13:14 dist-test-slave-1vmg krb5kdc[3231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.62@KRBTEST.COM
I20260811 15:13:14.640815 3630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.633179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.61:43357 (local address 127.0.0.1:42668)
0811 15:13:14.633446 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.633459 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:13:14.633538 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:13:14.633752 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:13:14.633754 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:13:14.633948 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:13:14.634044 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:14.634051 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.634857 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.634866 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:14.635444 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:14.635454 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.635922 (+ 468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.635924 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:14.635989 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.636484 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:13:14.636494 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:13:14.637750 (+ 1256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:13:14.639352 (+ 1602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:14.639355 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:14.639357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:14.639547 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:14.639809 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:14.639812 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:14.639814 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:14.639891 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:14.640313 (+ 422us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:13:14.640315 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:13:14.640414 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 15:13:14.640520 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 15:13:14.640665 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207}
I20260811 15:13:14.640887 3489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.633325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42668 (local address 127.30.8.61:43357)
0811 15:13:14.633449 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.633454 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.633499 (+ 45us) server_negotiation.cc:407] Connection header received
0811 15:13:14.633612 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.633615 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.633659 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.633724 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.634160 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.634730 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.635560 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.635813 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.635910 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.637881 (+ 1971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:14.637884 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:14.637886 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:14.637910 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:14.639252 (+ 1342us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.639636 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.639639 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.639641 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.639714 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.639994 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.639997 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.639998 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.640214 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:14.640319 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:14.640596 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 15:13:14.640730 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 15:13:14.641786 3626 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } has no permanent_uuid. Determining permanent_uuid...
I20260811 15:13:14.641844 3361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.632938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44004 (local address 127.30.8.62:44883)
0811 15:13:14.633077 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.633081 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.633092 (+ 11us) server_negotiation.cc:407] Connection header received
0811 15:13:14.633124 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.633127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.633174 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.633256 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.633947 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.634524 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.635429 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.635690 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.635766 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.638422 (+ 2656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:14.638425 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:14.638427 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:14.638451 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:14.639845 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.640229 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.640232 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.640234 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.640324 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.640588 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.640591 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.640593 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.640803 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:14.641025 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:14.641601 (+ 576us) server_negotiation.cc:299] Negotiation successful
0811 15:13:14.641724 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 15:13:14 dist-test-slave-1vmg krb5kdc[3231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461194, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.60@KRBTEST.COM for kudu/127.30.8.60@KRBTEST.COM
Aug 11 15:13:14 dist-test-slave-1vmg krb5kdc[3231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461193, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.8.61@KRBTEST.COM
I20260811 15:13:14.650058 3630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.642299 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37557 (local address 127.0.0.1:36400)
0811 15:13:14.642398 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.642411 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 15:13:14.642483 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:13:14.642756 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:13:14.642758 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:13:14.642961 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:13:14.643063 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:14.643070 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.643892 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.643896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:14.644434 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:14.644445 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.644866 (+ 421us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.644870 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:14.644953 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.645431 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:13:14.645443 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:13:14.646698 (+ 1255us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:13:14.648666 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:14.648668 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:14.648670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:14.648836 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:14.649113 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:14.649116 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:14.649118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:14.649192 (+ 74us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:14.649536 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:13:14.649539 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:13:14.649626 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 15:13:14.649713 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 15:13:14.649865 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 15:13:14.650060 3629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.642405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36400 (local address 127.30.8.60:37557)
0811 15:13:14.642575 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.642579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.642592 (+ 13us) server_negotiation.cc:407] Connection header received
0811 15:13:14.642624 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.642627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.642666 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.642743 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.643182 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.643784 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.644545 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.644773 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.644844 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.646816 (+ 1972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:14.646819 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:14.646821 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:14.646840 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:14.648562 (+ 1722us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.648923 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.648927 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.648929 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.649004 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.649268 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.649271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.649273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.649452 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:14.649536 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:14.649767 (+ 231us) server_negotiation.cc:299] Negotiation successful
0811 15:13:14.649919 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":107}
I20260811 15:13:14.650556 3489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.643086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42674 (local address 127.30.8.61:43357)
0811 15:13:14.643225 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.643229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.643241 (+ 12us) server_negotiation.cc:407] Connection header received
0811 15:13:14.643298 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.643300 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.643341 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.643413 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.643932 (+ 519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.644383 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.645230 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.645499 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.645586 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.647766 (+ 2180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:14.647769 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:14.647771 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:14.647794 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:14.649082 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.649457 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.649460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.649462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.649536 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.649784 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.649788 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.649790 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.649964 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:14.650168 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:14.650308 (+ 140us) server_negotiation.cc:299] Negotiation successful
0811 15:13:14.650419 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 15:13:14.653820 30752 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 15:13:14.654062 3626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900: Bootstrap starting.
I20260811 15:13:14.655017 3309 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "d6a5f051de274ab98b4bb5be1055769d"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44004
I20260811 15:13:14.656684 3626 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900: Neither blocks nor log segments found. Creating new log.
I20260811 15:13:14.657465 3626 log.cc:824] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900: Log is configured to *not* fsync() on all Append() calls
I20260811 15:13:14.659796 3626 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900: No bootstrap required, opened a new log
I20260811 15:13:14.664058 3626 raft_consensus.cc:348] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5f051de274ab98b4bb5be1055769d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } } peers { permanent_uuid: "56c6940e484e48e19c8680465145ecbc" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } } peers { permanent_uuid: "b3673b1715fd496d8fa081c853098900" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } }
I20260811 15:13:14.664355 3626 raft_consensus.cc:729] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3673b1715fd496d8fa081c853098900, State: Initialized, Role: FOLLOWER
I20260811 15:13:14.664723 3626 consensus_queue.cc:260] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5f051de274ab98b4bb5be1055769d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } } peers { permanent_uuid: "56c6940e484e48e19c8680465145ecbc" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } } peers { permanent_uuid: "b3673b1715fd496d8fa081c853098900" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } }
I20260811 15:13:14.665277 3632 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5f051de274ab98b4bb5be1055769d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } } peers { permanent_uuid: "56c6940e484e48e19c8680465145ecbc" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } } peers { permanent_uuid: "b3673b1715fd496d8fa081c853098900" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } } }
I20260811 15:13:14.665412 3632 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:13:14.666245 3626 sys_catalog.cc:564] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 15:13:14.672046 3647 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:13:14.672143 3647 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:13:14.670292 3501 master_runner.cc:186] Error getting master registration for 127.30.8.62:44883: OK, instance_id { permanent_uuid: "d6a5f051de274ab98b4bb5be1055769d" instance_seqno: 1786461194121876 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 15:13:14.673470 3501 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 15:13:14.743758 3481 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } attempt: 4
Aug 11 15:13:14 dist-test-slave-1vmg krb5kdc[3231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461194, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.61@KRBTEST.COM for kudu/127.30.8.60@KRBTEST.COM
I20260811 15:13:14.753780 3486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.744394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37557 (local address 127.0.0.1:36408)
0811 15:13:14.744571 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.744589 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 15:13:14.744700 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:13:14.744989 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:13:14.744993 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:13:14.745227 (+ 234us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:13:14.745347 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:14.745354 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.746373 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.746377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:14.747261 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:14.747274 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.747846 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.747850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:14.747941 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.748544 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:13:14.748560 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:13:14.750162 (+ 1602us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:13:14.752148 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:14.752152 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:14.752154 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:14.752408 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:14.752681 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:14.752684 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:14.752686 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:14.752778 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:14.753224 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:13:14.753226 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:13:14.753334 (+ 108us) client_negotiation.cc:769] Sending connection context
0811 15:13:14.753433 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 15:13:14.753592 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49}
I20260811 15:13:14.753787 3629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.744502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36408 (local address 127.30.8.60:37557)
0811 15:13:14.744655 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.744660 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.744680 (+ 20us) server_negotiation.cc:407] Connection header received
0811 15:13:14.744804 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.744807 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.744862 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.744948 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.745462 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.746259 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.747385 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.747739 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.747829 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.750330 (+ 2501us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:14.750335 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:14.750338 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:14.750369 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:14.752028 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.752506 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.752510 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.752512 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.752595 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.752858 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.752861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.752863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.753139 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:14.753233 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:14.753487 (+ 254us) server_negotiation.cc:299] Negotiation successful
0811 15:13:14.753621 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 15:13:14.757218 3481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc: Bootstrap starting.
I20260811 15:13:14.760620 3481 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc: Neither blocks nor log segments found. Creating new log.
I20260811 15:13:14.761729 3357 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } attempt: 4
I20260811 15:13:14.761991 3481 log.cc:824] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc: Log is configured to *not* fsync() on all Append() calls
I20260811 15:13:14.765163 3481 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc: No bootstrap required, opened a new log
Aug 11 15:13:14 dist-test-slave-1vmg krb5kdc[3231](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786461194, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.8.62@KRBTEST.COM for kudu/127.30.8.60@KRBTEST.COM
I20260811 15:13:14.770637 3481 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5f051de274ab98b4bb5be1055769d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } } peers { permanent_uuid: "56c6940e484e48e19c8680465145ecbc" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } } peers { permanent_uuid: "b3673b1715fd496d8fa081c853098900" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } }
I20260811 15:13:14.771034 3481 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56c6940e484e48e19c8680465145ecbc, State: Initialized, Role: FOLLOWER
I20260811 15:13:14.771498 3481 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5f051de274ab98b4bb5be1055769d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } } peers { permanent_uuid: "56c6940e484e48e19c8680465145ecbc" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } } peers { permanent_uuid: "b3673b1715fd496d8fa081c853098900" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } }
I20260811 15:13:14.772601 3650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5f051de274ab98b4bb5be1055769d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } } peers { permanent_uuid: "56c6940e484e48e19c8680465145ecbc" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } } peers { permanent_uuid: "b3673b1715fd496d8fa081c853098900" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } } }
I20260811 15:13:14.772842 3650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:13:14.773576 3481 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:13:14.773805 3365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.762495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.8.60:37557 (local address 127.0.0.1:36424)
0811 15:13:14.762762 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0811 15:13:14.762781 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 15:13:14.762954 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 15:13:14.763415 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 15:13:14.763420 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 15:13:14.763665 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0811 15:13:14.763823 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:14.763831 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.764948 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.764951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:14.765771 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 15:13:14.765785 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.766441 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.766444 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 15:13:14.766540 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.767241 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 15:13:14.767258 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 15:13:14.769181 (+ 1923us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 15:13:14.771509 (+ 2328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:14.771514 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:14.771516 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:14.771820 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:14.772239 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 15:13:14.772242 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 15:13:14.772244 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 15:13:14.772361 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 15:13:14.773100 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 15:13:14.773103 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 15:13:14.773224 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 15:13:14.773349 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 15:13:14.773542 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149}
I20260811 15:13:14.773823 3629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.762566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36424 (local address 127.30.8.60:37557)
0811 15:13:14.763131 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.763136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.763152 (+ 16us) server_negotiation.cc:407] Connection header received
0811 15:13:14.763197 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.763200 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.763247 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.763333 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.763996 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.764813 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.765920 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.766256 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.766376 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.769379 (+ 3003us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:14.769384 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:14.769387 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:14.769426 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:14.771371 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.771951 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.771955 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.771957 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.772064 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.772504 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.772509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.772511 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.772839 (+ 328us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:14.772928 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:14.773471 (+ 543us) server_negotiation.cc:299] Negotiation successful
0811 15:13:14.773646 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477}
I20260811 15:13:14.778270 3357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d: Bootstrap starting.
W20260811 15:13:14.781330 3664 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:13:14.781569 3664 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:13:14.781553 3357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d: Neither blocks nor log segments found. Creating new log.
I20260811 15:13:14.778805 3370 master_runner.cc:186] Error getting master registration for 127.30.8.62:44883: OK, instance_id { permanent_uuid: "d6a5f051de274ab98b4bb5be1055769d" instance_seqno: 1786461194121876 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 15:13:14.782473 3370 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 15:13:14.782543 3357 log.cc:824] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d: Log is configured to *not* fsync() on all Append() calls
I20260811 15:13:14.785455 3357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d: No bootstrap required, opened a new log
I20260811 15:13:14.788959 3357 raft_consensus.cc:348] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5f051de274ab98b4bb5be1055769d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } } peers { permanent_uuid: "56c6940e484e48e19c8680465145ecbc" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } } peers { permanent_uuid: "b3673b1715fd496d8fa081c853098900" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } }
I20260811 15:13:14.789264 3357 raft_consensus.cc:729] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d6a5f051de274ab98b4bb5be1055769d, State: Initialized, Role: FOLLOWER
I20260811 15:13:14.789675 3357 consensus_queue.cc:260] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5f051de274ab98b4bb5be1055769d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } } peers { permanent_uuid: "56c6940e484e48e19c8680465145ecbc" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } } peers { permanent_uuid: "b3673b1715fd496d8fa081c853098900" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } }
I20260811 15:13:14.791486 3357 sys_catalog.cc:564] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 15:13:14.791680 3309 raft_consensus.cc:482] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 15:13:14.791891 3309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:13:14.793776 3309 raft_consensus.cc:504] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5f051de274ab98b4bb5be1055769d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } } peers { permanent_uuid: "56c6940e484e48e19c8680465145ecbc" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } } peers { permanent_uuid: "b3673b1715fd496d8fa081c853098900" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } }
I20260811 15:13:14.795536 3309 leader_election.cc:288] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [CANDIDATE]: Term 1 election: Requested vote from peers 56c6940e484e48e19c8680465145ecbc (127.30.8.61:43357), b3673b1715fd496d8fa081c853098900 (127.30.8.60:37557)
I20260811 15:13:14.796347 3574 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6a5f051de274ab98b4bb5be1055769d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b3673b1715fd496d8fa081c853098900"
I20260811 15:13:14.796490 3435 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d6a5f051de274ab98b4bb5be1055769d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "56c6940e484e48e19c8680465145ecbc"
I20260811 15:13:14.796705 3574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900 [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:13:14.796717 3435 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc [term 0 FOLLOWER]: Advancing to term 1
I20260811 15:13:14.798383 3574 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a5f051de274ab98b4bb5be1055769d in term 1.
I20260811 15:13:14.798384 3435 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d6a5f051de274ab98b4bb5be1055769d in term 1.
I20260811 15:13:14.799420 3273 leader_election.cc:302] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3673b1715fd496d8fa081c853098900, d6a5f051de274ab98b4bb5be1055769d; no voters:
I20260811 15:13:14.800108 3701 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5f051de274ab98b4bb5be1055769d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } } peers { permanent_uuid: "56c6940e484e48e19c8680465145ecbc" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } } peers { permanent_uuid: "b3673b1715fd496d8fa081c853098900" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } } }
W20260811 15:13:14.800403 3690 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 15:13:14.800541 3690 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 15:13:14.800426 3701 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:13:14.801754 3701 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 15:13:14.801900 3701 raft_consensus.cc:686] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [term 1 LEADER]: Becoming Leader. State: Replica: d6a5f051de274ab98b4bb5be1055769d, State: Running, Role: LEADER
I20260811 15:13:14.802397 3701 consensus_queue.cc:237] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5f051de274ab98b4bb5be1055769d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } } peers { permanent_uuid: "56c6940e484e48e19c8680465145ecbc" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } } peers { permanent_uuid: "b3673b1715fd496d8fa081c853098900" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } }
I20260811 15:13:14.805006 3701 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6a5f051de274ab98b4bb5be1055769d. Latest consensus state: current_term: 1 leader_uuid: "d6a5f051de274ab98b4bb5be1055769d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5f051de274ab98b4bb5be1055769d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } } peers { permanent_uuid: "56c6940e484e48e19c8680465145ecbc" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } } peers { permanent_uuid: "b3673b1715fd496d8fa081c853098900" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } } }
I20260811 15:13:14.805245 3701 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [sys.catalog]: This master's current role is: LEADER
I20260811 15:13:14.807525 3252 master_runner.cc:263] No leader master found from master d6a5f051de274ab98b4bb5be1055769d
I20260811 15:13:14.807601 3252 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 15:13:14.817078 3715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 15:13:14.821272 3715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 15:13:14.828116 3574 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900 [term 1 FOLLOWER]: Refusing update from remote peer d6a5f051de274ab98b4bb5be1055769d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:13:14.829383 3670 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3673b1715fd496d8fa081c853098900" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:13:14.832019 3629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.804305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36438 (local address 127.30.8.60:37557)
0811 15:13:14.804451 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.804456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.815312 (+ 10856us) server_negotiation.cc:407] Connection header received
0811 15:13:14.815540 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.815543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.815603 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.815662 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.816772 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.817583 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.818802 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.819187 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.819266 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.821690 (+ 2424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:14.821695 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:14.821698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:14.821727 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:14.823514 (+ 1787us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.829480 (+ 5966us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.829484 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.829486 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.829585 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.830299 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.830303 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.830305 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.830540 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:14.830653 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:14.831681 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0811 15:13:14.831823 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 15:13:14.833328 3489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.802309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42676 (local address 127.30.8.61:43357)
0811 15:13:14.802677 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.802683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.811554 (+ 8871us) server_negotiation.cc:407] Connection header received
0811 15:13:14.811747 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.811751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.811817 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.811872 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.813117 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.813973 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.815232 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.815591 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.815667 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.824065 (+ 8398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:14.824069 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:14.824072 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:14.824105 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:14.825729 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.829893 (+ 4164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.829898 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.829899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.829995 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.830996 (+ 1001us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.830999 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.831001 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.831243 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:14.831348 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:14.833040 (+ 1692us) server_negotiation.cc:299] Negotiation successful
0811 15:13:14.833173 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267}
I20260811 15:13:14.833963 3632 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6a5f051de274ab98b4bb5be1055769d. Latest consensus state: current_term: 1 leader_uuid: "d6a5f051de274ab98b4bb5be1055769d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5f051de274ab98b4bb5be1055769d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } } peers { permanent_uuid: "56c6940e484e48e19c8680465145ecbc" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } } peers { permanent_uuid: "b3673b1715fd496d8fa081c853098900" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } } }
I20260811 15:13:14.834100 3632 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:13:14.835878 3435 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc [term 1 FOLLOWER]: Refusing update from remote peer d6a5f051de274ab98b4bb5be1055769d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 15:13:14.836529 3670 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [LEADER]: Connected to new peer: Peer: permanent_uuid: "56c6940e484e48e19c8680465145ecbc" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 15:13:14.837371 3361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:14.802782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44012 (local address 127.30.8.62:44883)
0811 15:13:14.802906 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.802910 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.811383 (+ 8473us) server_negotiation.cc:407] Connection header received
0811 15:13:14.811687 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.811691 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.811761 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.811826 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.823443 (+ 11617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.824176 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.825310 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:14.825664 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:14.825743 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:14.827348 (+ 1605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 15:13:14.827351 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 15:13:14.827354 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 15:13:14.827381 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 15:13:14.828892 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.829373 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.829376 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.829378 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.834686 (+ 5308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 15:13:14.836335 (+ 1649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 15:13:14.836339 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 15:13:14.836341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 15:13:14.836593 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 15:13:14.836724 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:14.837081 (+ 357us) server_negotiation.cc:299] Negotiation successful
0811 15:13:14.837218 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 15:13:14.837771 3403 tablet.cc:2369] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 15:13:14.840965 3670 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d6a5f051de274ab98b4bb5be1055769d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5f051de274ab98b4bb5be1055769d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } } peers { permanent_uuid: "56c6940e484e48e19c8680465145ecbc" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } } peers { permanent_uuid: "b3673b1715fd496d8fa081c853098900" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } } }
I20260811 15:13:14.841111 3670 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [sys.catalog]: This master's current role is: LEADER
I20260811 15:13:14.842422 3632 sys_catalog.cc:455] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d6a5f051de274ab98b4bb5be1055769d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5f051de274ab98b4bb5be1055769d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } } peers { permanent_uuid: "56c6940e484e48e19c8680465145ecbc" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } } peers { permanent_uuid: "b3673b1715fd496d8fa081c853098900" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } } }
I20260811 15:13:14.842555 3632 sys_catalog.cc:458] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:13:14.844015 3650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc [sys.catalog]: SysCatalogTable state changed. Reason: New leader d6a5f051de274ab98b4bb5be1055769d. Latest consensus state: current_term: 1 leader_uuid: "d6a5f051de274ab98b4bb5be1055769d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5f051de274ab98b4bb5be1055769d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } } peers { permanent_uuid: "56c6940e484e48e19c8680465145ecbc" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } } peers { permanent_uuid: "b3673b1715fd496d8fa081c853098900" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } } }
I20260811 15:13:14.844156 3650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:13:14.844339 3716 mvcc.cc:205] Tried to move back new op lower bound from 7317345054005547008 to 7317345053915987968. Current Snapshot: MvccSnapshot[applied={T|T < 7317345054005547008}]
I20260811 15:13:14.844504 3718 mvcc.cc:205] Tried to move back new op lower bound from 7317345054005547008 to 7317345053915987968. Current Snapshot: MvccSnapshot[applied={T|T < 7317345054005547008}]
I20260811 15:13:14.849540 3715 catalog_manager.cc:1347] Generated new cluster ID: 5436b4bf5d394888bf231c5d8338259a
I20260811 15:13:14.849634 3715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 15:13:14.862866 3701 sys_catalog.cc:455] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d6a5f051de274ab98b4bb5be1055769d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5f051de274ab98b4bb5be1055769d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } } peers { permanent_uuid: "56c6940e484e48e19c8680465145ecbc" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } } peers { permanent_uuid: "b3673b1715fd496d8fa081c853098900" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } } }
I20260811 15:13:14.863049 3701 sys_catalog.cc:458] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d [sys.catalog]: This master's current role is: LEADER
I20260811 15:13:14.864214 3650 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d6a5f051de274ab98b4bb5be1055769d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d6a5f051de274ab98b4bb5be1055769d" member_type: VOTER last_known_addr { host: "127.30.8.62" port: 44883 } } peers { permanent_uuid: "56c6940e484e48e19c8680465145ecbc" member_type: VOTER last_known_addr { host: "127.30.8.61" port: 43357 } } peers { permanent_uuid: "b3673b1715fd496d8fa081c853098900" member_type: VOTER last_known_addr { host: "127.30.8.60" port: 37557 } } }
I20260811 15:13:14.864353 3650 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc [sys.catalog]: This master's current role is: FOLLOWER
I20260811 15:13:14.885747 3715 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 15:13:14.887215 3715 catalog_manager.cc:1504] Loading token signing keys...
I20260811 15:13:14.896155 3715 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d6a5f051de274ab98b4bb5be1055769d: Generated new TSK 0
I20260811 15:13:14.897171 3715 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 15:13:14.925951 3361 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:14.923975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44014 (local address 127.30.8.62:44883)
0811 15:13:14.924229 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.924233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.924777 (+ 544us) server_negotiation.cc:407] Connection header received
0811 15:13:14.924989 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.924993 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.925060 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.925113 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.925804 (+ 691us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:44014: BlockingRecv error: recv got EOF from 127.0.0.1:44014 (error 108)
Metrics: {"server-negotiator.queue_time_us":170}
W20260811 15:13:14.926721 3629 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:14.923975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36450 (local address 127.30.8.60:37557)
0811 15:13:14.924122 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.924127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.925296 (+ 1169us) server_negotiation.cc:407] Connection header received
0811 15:13:14.925476 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.925479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.925530 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.925594 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.926598 (+ 1004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:36450: BlockingRecv error: recv got EOF from 127.0.0.1:36450 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 15:13:14.927254 3489 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:14.924656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42678 (local address 127.30.8.61:43357)
0811 15:13:14.924807 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.924811 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.926353 (+ 1542us) server_negotiation.cc:407] Connection header received
0811 15:13:14.926522 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.926526 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.926577 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.926627 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.927141 (+ 514us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42678: BlockingRecv error: recv got EOF from 127.0.0.1:42678 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
W20260811 15:13:14.930852 3489 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:14.929604 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42680 (local address 127.30.8.61:43357)
0811 15:13:14.929712 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.929716 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.929924 (+ 208us) server_negotiation.cc:407] Connection header received
0811 15:13:14.930157 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.930161 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.930244 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.930328 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.930751 (+ 423us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42680: BlockingRecv error: recv got EOF from 127.0.0.1:42680 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260811 15:13:14.934849 3489 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:14.933957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42684 (local address 127.30.8.61:43357)
0811 15:13:14.934090 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.934094 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.934108 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:13:14.934156 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.934159 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.934211 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.934287 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.934739 (+ 452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42684: BlockingRecv error: recv got EOF from 127.0.0.1:42684 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 15:13:14.940552 3489 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:14.939140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42686 (local address 127.30.8.61:43357)
0811 15:13:14.939276 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.939281 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.939671 (+ 390us) server_negotiation.cc:407] Connection header received
0811 15:13:14.939797 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.939801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.939858 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.939942 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.940435 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42686: BlockingRecv error: recv got EOF from 127.0.0.1:42686 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 15:13:14.948064 3489 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:14.946456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42690 (local address 127.30.8.61:43357)
0811 15:13:14.946600 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.946605 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.947146 (+ 541us) server_negotiation.cc:407] Connection header received
0811 15:13:14.947289 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.947293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.947355 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.947452 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.947942 (+ 490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42690: BlockingRecv error: recv got EOF from 127.0.0.1:42690 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260811 15:13:14.960209 3489 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:14.958482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42696 (local address 127.30.8.61:43357)
0811 15:13:14.958617 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.958622 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.959194 (+ 572us) server_negotiation.cc:407] Connection header received
0811 15:13:14.959394 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.959397 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.959450 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.959502 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.960079 (+ 577us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42696: BlockingRecv error: recv got EOF from 127.0.0.1:42696 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 15:13:14.980221 3489 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:14.979038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42698 (local address 127.30.8.61:43357)
0811 15:13:14.979184 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:14.979189 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:14.979374 (+ 185us) server_negotiation.cc:407] Connection header received
0811 15:13:14.979500 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:14.979503 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:14.979552 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:14.979624 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:14.980098 (+ 474us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42698: BlockingRecv error: recv got EOF from 127.0.0.1:42698 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 15:13:15.015822 3489 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:15.014651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42702 (local address 127.30.8.61:43357)
0811 15:13:15.014810 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:15.014814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:15.014873 (+ 59us) server_negotiation.cc:407] Connection header received
0811 15:13:15.015080 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:15.015083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:15.015131 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:15.015204 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:15.015689 (+ 485us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42702: BlockingRecv error: recv got EOF from 127.0.0.1:42702 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
W20260811 15:13:15.083132 3489 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:15.082079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42710 (local address 127.30.8.61:43357)
0811 15:13:15.082211 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:15.082215 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:15.082363 (+ 148us) server_negotiation.cc:407] Connection header received
0811 15:13:15.082439 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:15.082442 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:15.082489 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:15.082568 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:15.082997 (+ 429us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42710: BlockingRecv error: recv got EOF from 127.0.0.1:42710 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 15:13:15.215039 3489 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:15.213670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42712 (local address 127.30.8.61:43357)
0811 15:13:15.213905 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:15.213910 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:15.214028 (+ 118us) server_negotiation.cc:407] Connection header received
0811 15:13:15.214206 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:15.214210 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:15.214262 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:15.214337 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:15.214867 (+ 530us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42712: BlockingRecv error: recv got EOF from 127.0.0.1:42712 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
W20260811 15:13:15.475229 3489 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 15:13:15.473823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42720 (local address 127.30.8.61:43357)
0811 15:13:15.474017 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:15.474023 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:15.474143 (+ 120us) server_negotiation.cc:407] Connection header received
0811 15:13:15.474324 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:15.474327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:15.474379 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:15.474454 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 15:13:15.475096 (+ 642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42720: BlockingRecv error: recv got EOF from 127.0.0.1:42720 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 15:13:15.673776 3647 catalog_manager.cc:1259] Loaded cluster ID: 5436b4bf5d394888bf231c5d8338259a
I20260811 15:13:15.673880 3647 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900: loading cluster ID for follower catalog manager: success
I20260811 15:13:15.675895 3647 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900: acquiring CA information for follower catalog manager: success
I20260811 15:13:15.677191 3647 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b3673b1715fd496d8fa081c853098900: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 15:13:15.783305 3664 catalog_manager.cc:1259] Loaded cluster ID: 5436b4bf5d394888bf231c5d8338259a
I20260811 15:13:15.783412 3664 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc: loading cluster ID for follower catalog manager: success
I20260811 15:13:15.785224 3664 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc: acquiring CA information for follower catalog manager: success
I20260811 15:13:15.786536 3664 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 56c6940e484e48e19c8680465145ecbc: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 15:13:15.993116 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:15.989298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42724 (local address 127.30.8.61:43357)
0811 15:13:15.989538 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:15.989542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:15.989657 (+ 115us) server_negotiation.cc:407] Connection header received
0811 15:13:15.989780 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:15.989782 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:15.989923 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:15.990027 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:13:15.990235 (+ 208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:15.990788 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:15.991615 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:15.991837 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:15.991909 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:15.992235 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:13:15.992759 (+ 524us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:13:15.992855 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:15.992916 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 15:13:15.992971 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":160,"thread_start_us":74,"threads_started":1}
I20260811 15:13:15.999747 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:15.996240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36454 (local address 127.30.8.60:37557)
0811 15:13:15.996481 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:15.996485 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:15.996499 (+ 14us) server_negotiation.cc:407] Connection header received
0811 15:13:15.996587 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:15.996589 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:15.996710 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:15.996807 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:13:15.996956 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:15.997495 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:15.998286 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:15.998525 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:15.998602 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:15.998862 (+ 260us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:13:15.999391 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:13:15.999487 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:15.999513 (+ 26us) server_negotiation.cc:299] Negotiation successful
0811 15:13:15.999568 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":81,"threads_started":1}
I20260811 15:13:16.006479 3847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:16.002890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36468 (local address 127.30.8.60:37557)
0811 15:13:16.003016 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:16.003020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:16.003122 (+ 102us) server_negotiation.cc:407] Connection header received
0811 15:13:16.003248 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:16.003252 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:16.003303 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:16.003381 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:13:16.003710 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:16.004440 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:16.005479 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:16.005702 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:16.005789 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:16.006029 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:13:16.006117 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:13:16.006184 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:16.006275 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 15:13:16.006330 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 15:13:16.007190 3837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:16.003089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42728 (local address 127.30.8.61:43357)
0811 15:13:16.003225 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:16.003230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:16.003608 (+ 378us) server_negotiation.cc:407] Connection header received
0811 15:13:16.003784 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:16.003787 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:16.003842 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:16.003915 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:13:16.004151 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:16.004780 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:16.005853 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:16.006178 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:16.006253 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:16.006728 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:13:16.006816 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:13:16.006888 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:16.007007 (+ 119us) server_negotiation.cc:299] Negotiation successful
0811 15:13:16.007066 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 15:13:16.007692 3862 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 15:13:16.002670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44022 (local address 127.30.8.62:44883)
0811 15:13:16.002938 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 15:13:16.002943 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 15:13:16.003346 (+ 403us) server_negotiation.cc:407] Connection header received
0811 15:13:16.003695 (+ 349us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 15:13:16.003699 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 15:13:16.003868 (+ 169us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 15:13:16.003982 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 15:13:16.004302 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:16.005053 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:16.006103 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 15:13:16.006447 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 15:13:16.006544 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 15:13:16.006831 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 15:13:16.007264 (+ 433us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 15:13:16.007349 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 15:13:16.007472 (+ 123us) server_negotiation.cc:299] Negotiation successful
0811 15:13:16.007535 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":81,"threads_started":1}
I20260811 15:13:16.010541 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 3252
I20260811 15:13:16.031342 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 3370
I20260811 15:13:16.047626 30752 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskxgyPSN/build/debug/bin/kudu with pid 3501
2026-08-11T15:13:16Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3486 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6933 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (302482 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST